What's the most efficient way to grep for two completely separate things and assign the values to separate variables?

Question

CentOS 6.x

I want to take the output from curl, grep for two completely separate strings, and assign their respective values as variables. What is the most efficient way to do this (without writing the output to disk)?

Normally I would think of having a script with something like:

#!/usr/bin/env bash
foo1=$(curl https://domain.com/file.xml | grep string1)
foo2=$(curl https://domain.com/file.xml | grep string2)

But this ends up taking two passes and is horribly inefficient. Is there a better way? Hopefully a solution that involves fewer passes?

Solution 1

1. Grepping into 1 variable

Try this:

foo1=$(curl https://domain.com/file.xml | grep -E "string1|string2")

This will run curl 1 time and grep for occurrences of string1 or string2.

2. Grepping into 2 variables

If they're different variables then change tactics slightly. Capture the output of curl and then grep afterwards.

output=$(curl https://domain.com/file.xml)
foo1=$(echo "$output" | grep "string1")
foo2=$(echo "$output" | grep "string2")

3. Grepping into an array

You could also store the results in an array instead of separate variables.

output=$(curl https://domain.com/file.xml)
readarray foo < <(echo "$output" | grep "string1|string2")

This one is a little tricky to deal with if your results from the grep might not return results, since the results from "string2" might be the first or second item in the array, but I'm providing it here just as a demonstration of the approach.

4. Reading into vars from grep

Yet another method which makes use of the read command along with process substitution (<( ..cmd..)).

$ read -d"\n" foo1 foo2 \
   <(curl https://domain.com/file.xml | grep -E "string1|string2")

This again can be tricky if the search for "string1" returns nothing, causing any matches for "string2" to show up in $foo1. Also this approach tends to be less portable than either #2 or #3 above.

Solution 2

read var1 var2 <<CURLSED
$(curl $url | sed -nr 's/(regx1)\|(regx2)/"\1"'"$IFS"'"\2"/p')
CURLSED

Seriously, though, guy, there are like a million ways to skin this cat.

Comments

[Mike B]

Sorry! I had a typo. The variables are actually different. :-( Summary is now updated.

[mikeserv]

You might want to note that the process substitution you recommend is far from a portable option.

[slm]

@mikeserv - thanks for the feedback, updated.