Creating an array from a text file in Bash
Solution 1
Use the mapfile
command:
mapfile -t myArray < file.txt
The error is using for
-- the idiomatic way to loop over lines of a file is:
while IFS= read -r line; do echo ">>$line<<"; done < file.txt
See BashFAQ/005 for more details.
Solution 2
mapfile
and readarray
(which are synonymous) are available in Bash version 4 and above. If you have an older version of Bash, you can use a loop to read the file into an array:
arr=()
while IFS= read -r line; do
arr+=("$line")
done < file
In case the file has an incomplete (missing newline) last line, you could use this alternative:
arr=()
while IFS= read -r line || [[ "$line" ]]; do
arr+=("$line")
done < file
Related:
Solution 3
You can do this too:
oldIFS="$IFS"
IFS=$'\n' arr=($(<file))
IFS="$oldIFS"
echo "${arr[1]}" # It will print `A Dog`.
Note:
Filename expansion still occurs. For example, if there's a line with a literal *
it will expand to all the files in current folder. So use it only if your file is free of this kind of scenario.
Solution 4
Use mapfile or read -a
Always check your code using shellcheck. It will often give you the correct answer. In this case SC2207 covers reading a file that either has space separated or newline separated values into an array.
Don't do this
array=( $(mycommand) )
Files with values separated by newlines
mapfile -t array < <(mycommand)
Files with values separated by spaces
IFS=" " read -r -a array <<< "$(mycommand)"
The shellcheck page will give you the rationale why this is considered best practice.
Solution 5
You can simply read each line from the file and assign it to an array.
#!/bin/bash
i=0
while read line
do
arr[$i]="$line"
i=$((i+1))
done < file.txt
user2856414
Updated on January 23, 2020Comments
-
user2856414 over 4 years
A script takes a URL, parses it for the required fields, and redirects its output to be saved in a file, file.txt. The output is saved on a new line each time a field has been found.
file.txt
A Cat A Dog A Mouse etc...
I want to take
file.txt
and create an array from it in a new script, where every line gets to be its own string variable in the array. So far I have tried:#!/bin/bash filename=file.txt declare -a myArray myArray=(`cat "$filename"`) for (( i = 0 ; i < 9 ; i++)) do echo "Element [$i]: ${myArray[$i]}" done
When I run this script, whitespace results in words getting split and instead of getting
Desired output
Element [0]: A Cat Element [1]: A Dog etc...
I end up getting this:
Actual output
Element [0]: A Element [1]: Cat Element [2]: A Element [3]: Dog etc...
How can I adjust the loop below such that the entire string on each line will correspond one-to-one with each variable in the array?
-
Hugues over 8 yearsIs there any way to set
IFS
only temporarily (so that it recovers its original value after this command), while still persisting the assignment toarr
? -
Hugues over 8 yearsNote that filename expansion still occurs; e.g.
IFS=$'\n' arr=($(echo 'a 1'; echo '*'; echo 'b 2')); printf "%s\n" "${arr[@]}"
-
Jahid over 8 years@Hugues : yap, filename expansion still occurs. I will add that bit of info..thnks..
-
Hugues over 8 yearsSorry, I disagree.
IFS=... command
does not changeIFS
in the current shell. However,IFS=... other_variable=...
(without any command) does change bothIFS
andother_variable
in the current shell. -
Jahid over 8 years@Hugues : You are right again, sorry about that... Fixed it with save-and-reset way.
-
Hugues over 8 yearsThanks! This works; it's unfortunate that there is no simpler way as I like the
arr=
notation (compared tomapfile
/readarray
). -
fedorqui about 8 yearsSince this is being promoted as the canonical q&a, you could also include what is mentioned in the link:
while IFS= read -r; do lines+=("$REPLY"); done <file
. -
ericslaw about 7 yearsmapfile does not exist in bash versions prior to 4.x
-
glenn jackman about 7 yearsBash 4 is about 5 years old now. Upgrade.
-
hola about 6 yearsHow do you access the array?
-
De Novo over 5 yearsDespite bash 4 being released in 2009, @ericslaw's comment remains relevant because many machines still ship with bash 3.x (and will not upgrade, so long as bash is released under GPLv3). If you're interested in portability, it's an important thing to note
-
glenn jackman over 5 yearsSure, an OS may ship with older bash, but individuals can upgrade their own installations, or install bash 4 separately (with homebrew or the like).
-
ericslaw over 5 yearsmacos mojave bash reports as 3.2.57 (wow really apple?). I suspect work wont let me touch too many things on the laptop though :(
-
glenn jackman over 5 yearsYou should be able to use whatever tools you need to do your job.
-
De Novo over 5 yearsthe issue isn't that a developer can't install an upgraded version, it's that a developer should be aware that a script using
mapfile
will not run as expected on many machines without additional steps. @ericslaw macs will continue to ship with bash 3.2.57 for the foreseeable future. More recent versions use a license that would require apple to share or allow things they don't want to share or allow. -
Tatiana Racheva over 4 yearsI find it that I have to put parentheses around
IFS= read -r line || [[ "$line" ]]
for it to work. Otherwise, it works great! -
codeforester over 4 years@TatianaRacheva: isn't it that the semicolon that was missing before
do
?