Passing variables between scripts in a Jenkins pipeline

Question

I have a declarative Jenkins pipeline that looks like the following:

pipeline{
    agent {
        ...
    }
    stages {
        stage("Provision EC2 Instance and Set ENV Variable"){
            steps {
                dir("docker-image-builder"){
                    sh 'python ./provisonInstance.py'
                }
            }
        }
        stage("Configure EC2 Instance with Ansible"){
            steps {
                ansiblePlaybook('./ansible.playbook.yml') {
                    ansibleName('1.9.4')
                }
            }
        }
    }
}

In my first stage, I have a Python script that provisions an Amazon EC2 instance that will be used to create an AMI. In the second stage, I will then configure configure my instance before finally creating an AMI and terminating the instance. In order to do this, I need the instance-id of the EC2 instance created in the first stage. Is there an elegant way to pass values generated by a script between stages, and if so how?

The most obvious method that I can think of is to write the value to a file on the agent in the first stage, and read it in the second stage, but this does not seem like an elegant solution.

Solution

For Declarative Pipeline:

This question already has a highly-upvoted answer on StackOverflow:

If you want to use a file (since a script is the thing generating the value you need):

pipeline {
  agent { label 'docker' }
  stages {
    stage('one') {
      steps {
        sh 'echo hotness > myfile.txt'
        script {
          // trim removes leading and trailing whitespace from the string
          myVar = readFile('myfile.txt').trim()
        }
        echo "${myVar}" // prints 'hotness'
      }
    }
    stage('two') {
      steps {
        echo "${myVar}" // prints 'hotness'
      }
    }
    // this stage is skipped due to the when expression, so nothing is printed
    stage('three') {
      when {
        expression { myVar != 'hotness' }
      }
      steps {
        echo "three: ${myVar}"
      }
    }
  }
}

A second solution would be to write out output from your script directly or with writeFile and then later read it back in with readFile. However, this comes with the significant caveat that both the reading and writing must take place on the same node, so you can't use this to send data across nodes.

You can also can archive files centrally with archiveArtifact and retrieve them with copyArtifact (and then read in the file with readFile).

Finally, there are some answers to a near-identical question on StackOverflow that offer alternative solutions.

---

However, if you can, I would suggest using Scripted Pipeline instead, which allows a much simpler solution:

def String myVar

stage('my-first-stage') {
  myVar = sh(script: 'my-command', returnStdout: true)
}

stage('my-second-stage') {
  sh("my-other-command --var='${myVar}'")
}

(I also already answered this question for Scripted Pipeline over on ServerFault.)

---

It looks like some of these solutions have already been proposed in the comments, but you are concerned that they are not elegant enough. I'm not really sure what kind of elegance you're looking for, because I can't think of any way to simplify these answers in Jenkins-Pipeline-land.

Comments

[Admin]

Not a full answer as we use scripted, not declarative pipelines, but writing to a file is exactly what I ended up doing in our scripts.

[Admin]

This question has been answered here stackoverflow.com/questions/47155326/…

[Admin]

@RamKamath The answer to that question is a little different. I am trying to take output from a python script and pass it to a stage. While I think that part of the answer is to create a global environment variable, set it in the first stage, and read it in the second stage, it doesn't provide an elegant way to pass it from the python script at the stage level.