Value of variable set in 'while read' loop looses value outside the loop

Question

The following code is a simplified version of the real life task. Of course I know that I could achieve what is done in this simplified example with a if [ ... ]. This is only to demonstrate the surprising behaviour:

# /bin/bash
CURRD=dir1
IS_THIS=N
DATALINES=dir1
echo $DATALINES | while read DLINE
do
    echo IS_THIS1 ${IS_THIS}
    if [ "$DLINE" != "$CURRD" ]
    then
        echo $DLINE is not current dir
    else
        echo $DLINE is current dir
        IS_THIS=Y
        echo IS_THIS2 ${IS_THIS}
    fi
done
echo IS_THIS3 ${IS_THIS}

I would expect to get this output:

IS_THIS1 N
dir1 is current dir
IS_THIS2 Y
IS_THIS3 Y

Instead I get this:

IS_THIS1 N
dir1 is current dir
IS_THIS2 Y
IS_THIS3 N

So inside the loop the variable IS_THIS was set to Y. I expected that it keeps that value outside the loop. But that is obviously not the case. It falls back to the value it had outside. This is as if the code inside the while-loop was a space with local variables. What didn't I understand about bash shell scripts here?

I tried this in a bash 3.2.51(1) and a bash 4.1.11(2). Both show the same behaviour.

PS: As I couldn't solve the real life task with the while read loop, I split the input at the newline setting IFS to the newline and initializing an array variable with the input in parenthesis and then did a for loop over the elements of the array variable. So the task is solved, I'm just curious to learn what about the while read loop in bash scripting I did not understand correctly.

Solution 1

This answer given in the comment above by l0b0 is correct. I would have marked it as the correct answer, but it is a comment, so I can't:

There are many duplicates on USE, but they can be hard to find. Look at for example Appending to same array in various loops, only last values remain Bash 4

There are two solutions given there. For me actually only the one with the Here-document worked. For easier finding the answer, here is my example code above done with a Here-document as suggested in USE. It avoids the pipe and thus the sub-shell for the loop and so the variable IS_THIS is the same and the setting of the value is preserved:

# /bin/bash
CURRD=dir1
IS_THIS=N
DATALINES=dir1
while read DLINE
do
    echo IS_THIS1 ${IS_THIS}
    if [ "$DLINE" != "$CURRD" ]
    then
        echo $DLINE is not current dir
    else
        echo $DLINE is current dir
        IS_THIS=Y
        echo IS_THIS2 ${IS_THIS}
    fi
done <<< "$DATALINES"
echo IS_THIS3 ${IS_THIS}

Solution 2

Loops are run as a separate process so variables inside a loop are destroyed when the loop ends.

If you can use Python, it does keep variable changes inside loops.

currd = ["dir1"]
is_this = "n"
datalines = ["dir1"]

print is_this # checking variable

for lines in datalines:
    if lines != "".join(currd):
        print lines + " is not current dir"
    else:
        print lines + " is current dir"
        is_this = "y"

print is_this # checking if variable changed

Comments

[l0b0]

There are many duplicates on USE, but they can be hard to find. Look at for example Appending to same array in various loops, only last values remain Bash 4.

[Art Gertner]

could you suggest the solution to the problem?

[Chris]

Why don't you put "echo IS_THIS3 ${IS_THIS}" inside the loop and create a function to change the variable to IS_THIS=Y function changevalue { IS_THIS=Y }

[Markus]

I could use Python, but I don't want to for two reasons. First I'm more experienced using shell script than Python and thus more productive with it. Second these are just a few lines out of a larger script. Using Python would mean to rewrite more than just the few lines of the demonstration code above.

[Markus]

I tried the solution with the function 'changevalue'. It does not work. Instead it shows the same behaviour than the original code. Obviously bash considers the 'global' scope (in which bash variables are by default) to be the scope of the sub-shell opened for the loop with the pipe. And so the value 'Y' is lost again after the shell is back outside the loop.