While loop read multiple lines from a grep

Question

I'm writing a script in AIX 5.3 that will loop through the output of a df and check each volume against another config file. If the volume appears in the config file, it will set a flag which is needed later in the script. If my config file only has a single column and I use a for loop, this works perfectly. My problem, however, is that if I use a while read loop to populate more than one variable per line, any variables I set between the while and the done are discarded.

For example, assuming the contents of /netapp/conf/ExcludeFile.conf are a bunch of lines containing two fields each:

volName="myVolume"
utilization=70
thresholdFlag=0

grep volName /netapp/conf/ExcludeFile.conf | while read vol threshold; do
    if [ $utilization -ge $threshold ] ; then
        thresholdFlag=1
    fi
done

echo "$thresholdFlag"

In this example, thresholdFlag will always be 0, even if the volume appears in the file and its utilization is greater than the threshold. I could have added an echo "setting thresholdFlag to 1" in there, see the echo, and it'll still echo a 0 at the end.

Is there a clean way to do this? I think my while loop is being done in a subshell, and changes I make to variables in there are actually being made to local variables that are discarded after the done.

Solution

Zoredache pointed this out to me in chat, and poige mentioned it in his answer: this problem can be solved with a subshell.

When I had to change from a for loop that read a single variable from my grep at a time to a while read var1 var2 loop that allowed me to read in multiple variables, I was able to hang on to temporary variables I manipulated within the while loop by using parentheses to define an explicit subshell. Here's an example:

sum=0

grep volume configfile | head -n1 | (while read var1 var2; do
  let sum=var1+var2
done

echo "The sum is $sum.")

Without the parentheses, you will always echo a sum of 0. With them, you will echo the sum of the first two values in the first matching line of your grep.

Additionally, as Poige points out in another answer, you can use a subshell to populate an in-scope variable like this:

var=$( cat file | while read a b; do
    sum=a+b
    echo "$sum"
done)

echo "$var"

In this case, the value of var that you echo at the end would be the last sum you calculated in your loop, even though sum got destroyed at the end of the subshell.

Comments

[Zoredache]

fvue.nl/wiki/… tldp.org/LDP/abs/html/subshells.html

[poige]

Please re-edit the Subject, since your question has very litte to do with line reading, it's rather about "variables I set between the while and the done are discarded."

[Zoredache]

@poige, I think the current title is a better choice for the purposes of SEO and searches. Almost everytime I see this asked the title is like above.

[Basil]

Also, this problem comes up when you have to switch from a for loop to a while read loop because of multiple variables.

[voretaq7]

Related question on U&L -- unix.stackexchange.com/questions/5869/…

[poige]

@Zoredache, I don't agree. See "Related question" given by @voretaq7♦ — it's not so related for the purpose, or I'm missing the point heavily. )

[Basil]

@poige The problem I was trying to solve is the title of the question. If, for example, there were a way to make a for loop parse lines of stdin into separate key variables, that would be an answer. The while loop is only in the subject of the question because it's the only way to do this, however has this limitation.

[Basil]

You're correct, but there's not enough detail in here for a shell noob to really get at what you mean.

[poige]

@Basil, haven't you understood mine answer?

[Basil]

Not at first glance- now that I've looked up what happens if you use square brackets with a condition and then a logical operand like && with an action after it, it's a little clearer, but certainly not something that someone at my level would understand at first glance. That said, I do understand it now, and if you want me to, I can add a longer explanation before the code.

[poige]

@Basil, please don't make cosmetic changes like changing [ … ] && to if then fi — it doesn't make sense and what it really is just style. I prefer brevity, others may prefer long ways to say the same. There's also no need in chewing up everything enough to feed a toothless — others won't eat it.

[Basil]

fine, then your answer is too brief to be helpful, and I'll accept another one that lays it out easily for shell noobs.

[Wesley]

@poige Translation: "Please don't use ServerFault like it was intended to be used."

[poige]

@WesleyDavid, Ergh, where could someone read that ServerFault was intended to be powdered baby milk supply? It's kinda amazing how far would one go in stupidity to make technical things explained in such a way that even an idiot would understand it, why aren't you giving explanation of binary digits first on the every answer?