How to make bash built-in "read" ignore commented or empty lines?

Question

Is it possible in Android to convert or save as 2 or more captured photos into Multipage tiff file? I'm having a hard time searching saving .tiff file images in android. Any sample codes will be helpful. Thank you.

Solution 1

You don't need a tempfile to do this, and sed (or awk) are far more flexible in comment processing than a shell case statement.

For example:

configfile='/opt/myconfigfile.txt'
[ $# -gt 0 ] && [ -r "$1" ] && configfile="$1"

sed -e 's/[[:space:]]*#.*// ; /^[[:space:]]*$/d' "$configfile" |
    while read var1 var2 var3 var4; do
      # stuff with var1, etc.
    done

# Note: var1 etc are not available to the script at this
# point. They are only available in the sub-shell running
# the while loop, and go away when that sub-shell ends.

This strips comments (with or without leading whitespace) and deletes empty lines from the input before piping it into the while loop. It handles comments on lines by themselves and comments appended to the end of the line:

# full-line comment
# var1 var2 var3 var4
abc 123 xyz def # comment here

Calling sed or awk for tasks like this isn't "absurd", it's perfectly normal. That's what these tools are for. As for performance, I'd bet that in anything but very tiny input files, the sed version would be much faster. Piping to sed has some startup overhead but runs very fast, while shell is slow.

---

Update 2022-05-03:

Note that the variables (var1, var2, var3, etc) which are set in the while read loop will "go out of scope" when the while loop ends. The can only be used inside that while loop. The while loop is being run in a sub-shell because the config file is being piped into it. When that sub-shell dies, its environment goes with it - and a child process can not change the environment of its parent process.

If you want the variables to retain their values after the while loop, you need to avoid using a pipe. For example, use input redirection (<) and process substitution (<(...)):

while read var1 var2 var3 var4; do
  # stuff with var1, etc.
done < <(sed -e 's/[[:space:]]*#.*// ; /^[[:space:]]*$/d' "$configfile")

# remainder of script can use var1 etc if and as needed.

With this process substitution version, the while loop runs in the parent shell and the sed script is run as a child process (with its output redirected into the while loop). sed and its environment goes away when it finished, while the shell running the while loop retains the variables created/changed by the loop.

Solution 2

This works because read breaks everything on whitespace (IFS), so if var1 is empty or starts with '#', then skip it.

while read var1 var2 var3 var4; do
   case $var1 in
       ''|\#*) continue ;;         # skip blank lines and lines starting with #
   esac
   echo "var1: '$var1'"
   # stuff with var1, etc.
done < "${1:-default_config_file}"

Then input has to be redirected into the loop instead of to the while command list. The "${1:-default_config_file}" expands to the first command line parameter if not empty, otherwise expands to default_config_file you can also use variable expansion, etc in the default value string.

Because you're interested in minimizing pre-processing, I think this is equivalent, but also removes all comments:

while read line; do
    echo "${line%%#*}" | {
        read var1 var2 var3 var4
        [ -z "$var1" ] && continue
        # stuff with var1, etc.
        for i in 1 2 3 4; do eval echo "\"var$i: \$var$i\""; done  #debug only!
    }
done < "${1:-default_config_file}"

This uses the shell parameter expansion substring processing feature. ${line%%#*} expands to the original value of line except the first # and everything after it are removed. Load that into var1-4 and continue as usual. The test to continue shortens because we now only need to check for empty string, not #.

Solution 3

You can do that without create temp file. The grep command will filter empty and commented lines.

while read var1 var2 var3; do
    echo $var1
    echo $var2
    echo $var3
    echo "etc..."
done < <(grep -v "^#\|^$" /opt/myconfigfile.txt)

Comments

[Admin]

The error is that the script redirects to the while command list instead of redirecting into the body of the loop (see my answer below). I think your default myconfigfile.txt code will work, see my answer for a more concise (and maybe less obvious) solution to overriding the default via a reference to $1, however my answer does not check the file for readability.

[Admin]

There are advantages for not silently reverting to the default if the filename in $1 is unreadable. The script would behave as if $1 was processed, while actually using the default. Depending on the context, you might want to use the $1 value and let the shell print the error when the file cannot be opened for the redirection.

[Wildcard]

This is perfect. No external dependencies (no additional processes created), just as requested. One more part to the question, though—the config I want to read is contained in a specific file. I want to softcode that into the script, allowing for overrides. I've clarified this part of the question above.

[Wildcard]

The script that handles the config file is going to be run automatically every few minutes. It seemed strange to me to be re-stripping out comments on the same file every few minutes. But with the pipe set up this way, it does handle the other part of my question also, so perhaps this way is best. Thanks.

[Alessio]

${1:-default} is useful but @wildcard wanted to test if "$1" was readable ([ -r "$1" ]) before overriding the default.

[Alessio]

any other program that reads in a config file is going to be processing comments on every run too. if it bothers you, then strip the comments and redirect to, say, "$configfile.nocomments" only if "$configfile" is newer than "$configfile.nocomments", and use "$configfile.nocomments" as input to the while loop.

[RobertL]

@cas Thank you. That's what I get for updating based on comments before reading the updated question... After reflection, I commented on the question about a problem when silently reverting to default.

[Alessio]

i was going to change it to that myself until i re-read the question :) silently reverting is probably bad, but the script could warn or exit if "$1" isn't readable.

[Jarek]

Somehow I like this way most :-)