How do I assign the output of a command into an array?

Question

I need to assign the results from a grep to an array... for example

grep -n "search term" file.txt | sed 's/:.*//'

This resulted in a bunch of lines with line numbers in which the search term was found.

1
3
12
19

What's the easiest way to assign them to a bash array? If I simply assign them to a variable they become a space-separated string.

Solution 1

To assign the output of a command to an array, you need to use a command substitution inside of an array assignment. For a general command command this looks like:

arr=( $(command) )

In the example of the OP, this would read:

arr=($(grep -n "search term" file.txt | sed 's/:.*//'))

The inner $() runs the command while the outer () causes the output to be an array. The problem with this is that it will not work when the output of the command contains spaces. To handle this, you can set IFS to \n.

IFS=$'\n' arr=($(grep -n "search term" file.txt | sed 's/:.*//'))

You can also cut out the need for sed by performing an expansion on each element of the array:

arr=($(grep -n "search term" file.txt))
arr=("${arr[@]%%:*}")

Solution 2

Space-separated strings are easily traversable in bash.

# save the ouput
output=$(grep -n "search term" file.txt | sed 's/:.*//')

# iterating by for.
for x in $output; do echo $x; done;

# awk
echo $output | awk '{for(i=1;i<=NF;i++) print $i;}'

# convert to an array
ar=($output)
echo ${ar[3]} # echos 4th element

if you are thinking space in file name use find . -printf "\"%p\"\n"

Comments

[jordanm]

But broken for files with spaces.

[Shiplu Mokaddim]

@jordanm use find . -printf "\"%p\"\n"

[tuxErrante]

to me it worked only without outer brackets dvmList=`grep -RohP '(?<=oramds:).+\.dvm' myFolder `

[jordanm]

@bastaPasta That is not not an array assignment. Array assignment may not have worked for you if you were not using a shell that supported arrays.

[tuxErrante]

You're right I was using sh instead of bash, but it works fine looping in a for

[jordanm]

@bastaPasta yes it will, as long as your "elements" do not contain spaces. There are other ways to simulate arrays for shells that do not have them via set and the special array $@.

[Charles Duffy]

Note that array=( $(command) ) is considered an antipattern, and is the topic of BashPitfalls #50.

[Charles Duffy]

@ShipluMokaddim, that doesn't work. for x in $output with such a find command will cause a file named two words to be iterated over as one value "two and another value words".

[Charles Duffy]

@ShipluMokaddim, ...the same thing is true if you use ar=( $output ) with output assigned in that way, because parameter expansion and command substitution results don't go through the quote removal parsing phase.

[Charles Duffy]

As a better-practice approach, still compatible with bash 3.x, consider read -r -d '' -a arr < <(grep ... | sed ... && printf '\0'), which avoids globbing and string-splitting your command's output, and also passes through a nonzero exit status from the pipeline (coalescing them all to 1, because the lack of a trailing delimiter makes read fail)

[Magne]

NB: If you have IFS=$'\n' set in general or for all of these lines (to handle terms with whitespace) then the append command will not keep the original array structure. It will just make array contain only 1 item (a string list), with all the changed strings. As seen if you run: declare -p array afterwards.

[levigroker]

I'm seeing an extraneous whitespace character at the end of every element of my array when using the IFS=$'\n' approach. Curious if I'm alone in this or if there's a known solution.

[jordanm]

@levigroker my guess is dos line endings, which are \r\n instead of just \n

[levigroker]

@jordanm Thanks for the reply. Coming back to this today has helped me find the root cause, which is not related to setting the IFS or the proposed solution. The issue was a dangling space in my input which I did not account for in my regular expression upstream of this. (for anyone curious, beware agvtool leaves trailing whitespace on every line of output)