Pass argument to script, then redirect script as input to bsub

Question

I am currently working with the bsub job submission system from Platform LSF. I need to submit job scripts, but I am running into a problem with passing arguments to the job script.

What I need is as follows: the job script is a bash script that takes one command line argument. I need to pass the argument to that script, then redirect the script to be used as input to bsub. The problem is that the argument isn't being applied to the script.

I have tried: bsub < script.bsub 1

script.bsub is my script and 1 is the numeric argument. This approach doesn't work. The 1 is being treated as an argument to bsub.

Another approach: bsub < "script.bsub 1" This approach makes bsub treat the entire double-quoted line as a filename.

Is there a way to apply a command line argument to a script first, then redirect that script with the argument into another program as input? If anyone has experience with bsub that would be even better.

UPDATE AND WORKAROUND:

It seems applying a command line argument to a bsub file is a very complicated process. I tried the HEREDOC method stated below, but bsub acted as if the script filename was a command. So the easiest way to get around this problem is just to not use input redirection at all.

To pass an argument to a script to be run in bsub, first specify all bsub arguments in the command line rather than in the script file. Then to run the script file, use

"sh script.sh [arg]"

after all of the bsub arguments. Thus the entire line will look something like:

bsub -q [queue] -J "[name]" -W 00:10 [other bsub args] "sh script.sh [script args]"

This doesn't answer my original question, but at least with bsub this is the simplest resolution to my problem.

Solution 1

It seems applying a command line argument to a bsub file is a very complicated process. I tried the HEREDOC method stated by mikeserv, but bsub acted as if the script filename was a command. So the easiest way to get around this problem is just to not use input redirection at all.

Since my question specifically involved bsub for Platform LSF, the following is probably the best way to solve this sort of argument problem: To pass an argument to a script to be run in bsub, first specify all bsub arguments in the command line rather than in the script file. Then to run the script file, use

"sh script.sh [arg]"

after all of the bsub arguments. Thus the entire line will look something like:

bsub -q [queue] -J "[name]" -W 00:10 [other bsub args] "sh script.sh [script args]"

In this case, it is better to not use .bsub files for the script and use a normal .sh script instead and use the Unix sh command to run it with arguments.

Solution 2

You need either a here document or a here string.

bsub <<HEREDOC
    $(script.bsub 1)
HEREDOC

That's what a here document looks like. The shell will read in the expansion of everything between the line in which <<HEREDOC occurs until it finds a line on which nothing but HEREDOC occurs and pass the output to the invoked command on its file descriptor 0 - stdin.

A here string is the same thing in principle:

bsub <<< script.bsub 1

It may look simpler to use, but it's also easier to confuse and it is not portable shell script. While a here document is defined as part of the POSIX shell standard, a here string is a shell specific implementation - though it will work in bash.

It is probably best to use a here document in every scripted case unless you can be 100% certain that the script will never be run in an environment in which it doesn't break, and to use here-strings only in interactive use as they are little more than a shortcut in the first place.

Of course, if either of those work, then probably:

script.bsub 1 | bsub

would do as well. That will |pipe script.bsub's stdout into bsub's stdin, but it will also very likely run bsub in a subshell. If bsub modifies the current shell environment in any way - such as altering the value of a shell variable - that is depended upon, the |pipe may not be the best solution.

Comments

[mikeserv]

Maybe put the answer part of your question in an answer and accept it? Also, does -- make any difference in a bsub command line?

[mikeserv]

Cool. Thank you. It gave me something to upvote again.