Transferring Build Artifacts between jobs in Jenkins.

The purpose of this article is to help anyone transfer their artifacts or let's say files between jobs in a Jenkins pipeline.

What exactly are Build Artifacts, according to JetBrains, these are files created by build processes such as distribution packages, WAR files, logs, and reports.

CICD pipelines typically do not transfer files automatically from one job to another. When a job finishes in a pipeline, any files created during the build are often destroyed, making them inaccessible to subsequent jobs. To address this limitation, it becomes necessary to establish a mechanism for transferring files between jobs.

In scenarios where files generated in one job are required in another job within the same pipeline, the Copy Artifact plugin comes into play in Jenkins. The Copy Artifact plugin provides a solution for copying artifacts from one job to another, enabling the seamless transfer of files.

By utilizing the Copy Artifact plugin, you can ensure that files generated in a specific job are preserved and made available for use in subsequent jobs.

The steps need to transfer your file between jobs are outlined below.

1. Install the copyartifact plugin.

   These are the installation options;

   a. Using the GUI: From your Jenkins dashboard navigate to Manage Jenkins > Manage Plugins and select the Available tab. Locate this plugin by searching for copyartifact.

   b. Using the CLI tool: jenkins-plugin-cli --plugins copyartifact:705.v5295cffec284

   c. Using direct upload. Download one of the releases and upload it to your Jenkins instance.

2. Configure your Pipeline.

   On your created pipeline navigate to Configure > Select Permission to Copy Artifact > In the space that shows type in the Name of the Pipeline Project.

   Let's say the name of the pipeline is test, type in test in the space.

3. Create your file to be transferred.

   Let's say you have a stage that creates a particular file called text.txt

    stage ("Create test file for transfer") {
       steps {
                sh 'echo "Hello World!" > text.txt'
                archieveArtifacts artifacts: 'text.txt'
           }
       }
   

   The archieveArtifacts command is responsible for achieving the created file so it can be used by other jobs.

4. Copy to the Job where the file is needed.

   The last step is to copy the archived file to the job where the file is needed.

     stage ("Use test file") {
      steps {
              copyArtifacts filter: 'text.txt', fingerprintArtifacts: true, projectName: 'test', selector: specific ("${BUILD_NUMBER}")
            sh 'cat "text.txt"'
       }
   }

copyArtifacts is the command provided by the copyartifact plugin installed earlier needed for retriving the archived file. The archived file to be used is specified in the filter value. It is important to note, the projectName value should be the same as what was filled in the space that was prompted during the configuration, i.e the projectName should be the name of the pipeline.

The output of sh 'cat "text.txt"' command will be Hello World! signifying it was properly copied.

An excellent application of artifact transfer is demonstrated in a pipeline that integrates Infracost and Terraform. In this scenario, a Terraform plan is generated and saved as a file within a specific job. This generated plan file is then transferred to the Infracost job, which utilizes the file to provide a cost estimate for your Terraform resources.

The pipeline workflow could be structured as follows:

1. Terraform Plan Job: This job performs the necessary Terraform actions, such as initialization and plan generation. The resulting Terraform plan is saved as a file.

2. Artifact Transfer: The Terraform plan file generated in the previous job is archived as an artifact.

3. Infracost Job: The Infracost job retrieves the Terraform plan artifact from the previous job using the artifact transfer mechanism. The retrieved plan file is then utilized by Infracost to calculate the cost estimate for your Terraform resources.