Copy file from Jenkins master to slave in Pipeline

Question

I have some windows slave at my Jenkins so I need to copy file to them in pipeline. I heard about Copy To Slave and Copy Artifact plugins, but they doesn't have pipeline syntax manual. So I don't know how to use them in pipeline.

Direct copy doesn't work.

def inputFile = input message: 'Upload file', parameters: [file(name: 'parameters.xml')]
new hudson.FilePath(new File("${ENV:WORKSPACE}\\parameters.xml")).copyFrom(inputFile)

This code returns and error:

Caused: java.io.IOException: Failed to copy /var/lib/jenkins/jobs/_dev/jobs/(TEST)job/builds/107/parameters.xml to d:\Jenkins\workspace\_dev\(TEST)job\parameters.xml

Is there any way to copy file from master to slave in Jenkins Pipeline?

Solution 1

As I understand copyFrom is executed on your Windows node, therefore the source path is not accessible.

I think you want to look into the stash/unstash steps (Jenkins Pipeline: Basic Steps), which work across different nodes. Also this example might be helpful.

Solution 2

Pipeline DSL context runs on master node even that your write node('someAgentName') in your pipeline.

• Try to use stash/unstash, but it is bad for large files.
• Try External Workspace Manager Plugin. It has pipelines steps and good for large files.
• Try to use an intermediate storage. archive() and sh("wget $url") will be helpful.

Comments

[Pavel Polushin]

Thank you very much. It helps.

[Chris F]

stash/unstash only works per job run, that is, the "stashed" data doesn't persist after the job runs.

[timblaktu]

Specifically, how is stash bad for large files? How would you define "large"? I've read this on the cloudbees docs as well, but they don't answer my questions above. What's the deal, is it just a slow transfer, or is it buggy?? Just curious why I shouldn't stash a 150MB file on a master so I can unstash it from a slave agent in a subsequent stage. Seems much simpler than using External Workspace Manager.

[timblaktu]

Sorry, apparently I didn't RTFM closely enough: the stash step pipeline reference says "This is because stashed files are archived in a compressed TAR, and with large files this demands considerable on-master resources, particularly CPU time. There's not a hard stash size limit, but between 5-100 MB you should probably consider alternatives."