How to pass in multiple arguments to execute with .sh script

Question

I have a list.txt file including a list of log file.

For example

    server_log1
    server_log2
    ........
    server_log50

I have another shell script used to download these logs. It worked like this

./script.sh serverlog1

I want to make it automatically that means it can automatically pass in each log file's name in list.txt to be executed.

Is that possible? I tried

#!/bin/bash
for i in `cat /home/ec2-user/list.txt` ; do
sh ./workaround.sh $i
done

But it didn't work

Solution 1

The easiest method for reading arguments can be described as follows;

Each argument is referenced and parsed by the $IFS or currently defined internal file separator. The default character is a space.

For example, take the following; # ./script.sh arg1 arg2

The argument list in that example is arg1 = $1 and arg2 = $2 which can be rewritten as arg1 arg2 = $@.

Another note is the use of a list of logs, how often does that change? My assumption is daily. Why not use the directory output as the array of your iterative loop? For example;

for i in $(ls /path/to/logs); do
  ./workaround.sh $i;
done

Or better yet, move on to use of functions in bash to eliminate clutter.

function process_file()
{
  # transfer file code/command
}

function iterate_dir()
{
  local -a dir=($(ls $1))
  for file in ${dir[@]}; do
    process_file $file
  done
}

iterate_dir /path/to/log/for

While these are merely suggestions to improve your shell scripting knowledge I must know if there is an error you are getting and would also need to know the details of each scripts code and or functionality. Making the use of the -x argument helps debug scripting as well.

If you are simply transferring logs you may wish to do away with the scripts all together and make use of rsync, rsyslog or syslog as they all are much more suited for the task in question.

Solution 2

xargs -n 1 ./script.sh <list.txt

In this example, xargs will execute ./script.sh multiple times with a single argument read from its standard input.

The standard input comes from the file list.txt via a simple shell input redirection. This assumes that the list.txt file contains one argument to use with the script per line.

The ./script.sh script will be executed once for each line in the list.txt file.

---

About the -n flag to xargs

-n number

Invoke utility using as many standard input arguments as possible, up to number (a positive decimal integer) arguments maximum.

---

Another solution would be to allow the script to read directly from list.txt, or to take multiple command line argument (the entries from list.txt), and to download the files in one or several batches. But since we don't know what the mechanics of the script is, I can't make any detailed explanation of how to go about doing this.

Solution 3

(Note: I personally think Kusalananda's approach is the best in this specific scenario, so I'll just add some explanatory information and recommendations)

---

Don't read lines with for

Using that approach you:

• Rely on word splitting to make things work.
• Are subject of unintended side effects like globbing.
• Slurp the entire file into memory all at once (may be a problem with big files).

The while + read approach is preferable but it is not a bullet-proof solution:

while IFS= read -r line; do
  # Your code here
done < file

Quote your variables

Write "$i" instead of $i. Unquoted variables are probably the main source of bugs and security holes in shell scripts.

Related questions:

• Why does my shell script choke on whitespace or other special characters?
• Security implications of forgetting to quote a variable in bash/POSIX shells

Use $(...) instead of `...`

$(...) is POSIX-compliant and can be nested easier.

---

You could also modify your script to support multiple arguments and even a --batch option.

If you don't know where to start, I have a sample/template script which supports those features (it uses Bash-specific syntax, though).

Example of usage:

$ cat list.txt
list_item_1
list_item_2
list_item_3
$ script --batch list.txt item_1 item_2
Operands:
  1: [list_item_1]
  2: [list_item_2]
  3: [list_item_3]
  4: [item_1]
  5: [item_2]

Solution 4

inside the script : you need a read loop like while read ; do ......... ; done < filename to treat lines as $REPLY variable...

for example

while read 
do
mv $REPLY $REPLY.old
done < liste.txt

will rename any filename from inside liste.txt to filename.old

you have the structure now you can adapt to your needs depending on what you mean by " in each log file's name in list.txt to be executed." :)

View more solutions

Comments

[Admin]

sidenote: don't use file extensions for executables (scripts etc). If you re-write in python, C, or what ever, then you don't want to have to change the file name, as it will cause you to change every script that uses it.

[The One]

Can you please check my latest code?

[Centimane]

You might want to explain the syntax you use a bit in your answer, since this seems like a bit of a newbie asking. Otherwise great answer, I'd say this is probably the best way to do it VS while read

[Kusalananda]

@Centimane Are you referring to the redirection, or to the semantics of -n 1? There's not much of syntax to explain...

[Centimane]

I would do both, but maybe that's just me. I'd quote the man page for the -n option and a sentence or two on the redirection.

[Josh]

$@ is what I was looking for. Thanks.