How to while read line with a pipe

Question

I am currently trying read a csv file with BASH. The 10th [$f10] value of the csv file must be greater then 0 for a given machine class and machine ID [m_class] and [m_id] (these are user defined) . I am tacing the file as the most recent information is written last. Here is the snippet of code [ignore the echos, as I placed them there too see if my code was entering the while and if loops]:

IFS=","
while read f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15
do

            echo "this is while loop"

            if [ "$f7" == "EnHr" ] && [ "$f10" -gt 0 ] && [ "$f4" == "$m_class" ] && [ "$f5" == "$m_id" ]
            then
                    hex_sensor_data=$f10
                    echo "IN IF LOOP: $hex_sensor_data"
                    break
            fi  
            echo "HEX DATA $hex_sensor_data"
done <<<"tac /DIRECTORY/TO/FILE.csv | sed 's/o,632/o,101/g' | sed 's/o,938/o,103/g' |  sed 's/o,510/o,112/g' | sed 's/ombo,713/ombo,102/g'"

This while loop renders a 0 value for any m_class m_id, also the $f10 is in hex value - After breaking from the while read loop I want to do some processing with hex_sensor_data value.

Q: I want to get the last value which is greater then 0 in the 10th space of the csv file whilst the 7th value of the csv holds EnHr, as well as the 4th and 5th hold for a given m_class and m_id respectively. To do so I need to read the csv file from bottom (latest information) to the top (earliest information) I am doing this with tac as well as doing some piping. Any help is appreciated!

Solution 1

<<< in bash is a here string. You passed your script string itself tac /DIRECTORY/TO/FILE.csv ... to while's stdin.

You might have wanted to use process substitution:

while read f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15
do
    ...
done < <(tac /DIRECTORY/TO/FILE.csv ...)

Solution 2

you can try

tac /DIRECTORY/TO/FILE.csv |
sed -e 's/o,632/o,101/g' -e  's/o,938/o,103/g'  -e 's/o,510/o,112/g' -e 's/ombo,713/ombo,102/g' |
while read f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13 f14 f15
do

            if [ "$f7" = "EnHr" ] && [ "$f10" -gt 0 ] && [ "$f4" = "$m_class" ] && [ "$f5" = "$m_id" ]
            then
                    hex_sensor_data=$f10
                    echo "IN IF LOOP: $hex_sensor_data"
                    break
            fi
done

• there is no need to backslah newline if last char is a pipe ( | )
• a single sed can run all substitution

Solution 3

#!/bin/bash

m_class='aaa'
m_id='bbb'

f4=0
f5=1
f7=2
f10=3

csvdata=( $(tac CSVFILE) )

hex_sensor_data=0
for linex in ${csvdata[@]}
do
    csvline=(
        $(echo "${linex}"|awk -F, '{if ($10 -gt 0) print $4,$5,$7,$10}')
    )
    [[ ${csvline[f4]} != "${m_class}" ]] && continue
    [[ ${csvline[f5]} != "${m_id}" ]] && continue
    [[ ${csvline[f7]} != "EnHr" ]] && continue
    hex_sensor_data=${csvline[f10]}
    break
done
[[ "${hex_sensor_data}" != "0" ]] && echo "${hex_sensor_data}"

Comments

[yaegashi]

You cannot refer $hex_sensor_data after the loop because bash sets it in subshell, in contrast to which zsh allows that reference.