Bash iterate over a list of strings

Question

I have this bash script:

for opt in string1 string2 string3 ... string99
do somestuff

It works, but I would like to replace the explicit listing of my strings with a file which actually contains all the strings; something like this:

strings=loadFromFile
for opt in $strings
do somestuff

How should I do this?

Solution 1

while read VAR is probably best here, as it handles per-line input. You can redirect it from a file, e.g.:

while IFS= read -r THELINE; do
  echo "..$THELINE"
done </path/to/file

That'll give you each line prepended with ".."

For your example case:

while IFS= read -r opt; do
  #somestuff $opt
done </path/to/file

See Why is `while IFS= read` used so often, instead of `IFS=; while read..`? for explanations.

Solution 2

while IFS= read -r opt
do 
    some_stuff
done < file_with_string

See Why is `while IFS= read` used so often, instead of `IFS=; while read..`? for explanations.

Solution 3

The while IFS= read -r line; do ...; done < aFile is the best answer

If your strings do not contain whitespace or \[*?, you could do

for word in $(< aFile); do something; done

$(< file) is a bash feature that reads the file (like cat except without having to spawn a new process).

Solution 4

Why don't you use the readarray builtin (requires bash >= 4.0)?

readarray < FileNameFromWhereToLoad    # push every line of 
                                       # 'FileNameFromWhereToLoad' onto 
                                       # $MAPFILE by default
for i in $MAPFILE ; do
    echo $i
done

Solution 5

My advice:

cat INPUTFILE| {
  declare -a LINES
  mapfile -t LINES
  for line in "${LINES[@]}"
  do
    somestuff
  done
}

View more solutions

Comments

[Gilles 'SO- stop being evil']

See Why is while IFS= read used so often, instead of IFS=; while read..? and In while IFS= read.., why does IFS have no effect?

[michelemarcon]

Syntax error: bad substitution

[manatwork]

Works for me. @michelemarcon, is your bash 4.00 or never?

[michelemarcon]

GNU bash, version 2.05a.0(1)-release (arm-unknown-linux-gnu)

[manatwork]

That is ancient. Please specify this requirement in the question.

[Razzlero]

This also requires newer version of bash.

[Razzlero]

Putting cats into pipes is bad behavior from a memory management point of view. especially when using "filter" built-in commands. @rush's answer is better, and what I use on a daily basis.

[user1146332]

i wouldn't call bash >= 4 new (release date was February 2009)!

[Razzlero]

Since he's using 2.05a, it is newer than what the asker has. In addition, some companies have avoided GPLv3 versions of bash, so do not have it in their system.

[user1146332]

Do you have a crystal ball around? But yes, i agree with you about your last statement, i will add the version requirement of readarray to my answer.

[Razzlero]

Yes, I have a crystal ball which reflects the comment the poster wrote an hour ago on @h-dirk-schmitt answer.

[user1146332]

I didn't read that. Thanks for the hint ;) Maybe my answer is of value for somebody who will search for an answer for a similar question!?

[Angel Todorov]

It also puts the while loop into a subshell, which can cause confusing behaviour if update variable values in the loop (the values will disappear when the subshell exits)

[Raphael]

If somestuff includes another read, this seems not to work properly.

[IBBoard]

@raphael - in what way does it not "work properly"? I just tried nested reads and it works fine.

[Raphael]

@IBBoard See this new question.

[AnneTheAgile]

osx 10.12.2 has ancient bash ; $ bash --version GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin16) Copyright (C) 2007 Free Software Foundation, Inc.