Bash script that reads filenames from a pipe or from command line args?

Question

I want my script to read a bunch of filenames (which may have spaces) given either as a glob or from STDIN and do stuff with them. I've been able to read either way separately, but not combine them.

This reads globs from the command line:

for filename in "$@"; do
    process_file "$filename"
done

And this reads from STDIN:

IFS=$'\n' read -d '' -r -a filenames
for filename in "${filenames[@]}"; do
    process_file "$filename"
done

What I really want is to read from either one into an array so I don't have to duplicate my entire for filename in... loop twice. Sorry if this is obvious, I'm new to BASH.

EDIT: I think the best thing would be to read from args if they're given, and otherwise wait for STDIN. How would I do that?

EDIT: OK, the problem isn't what I thought it was. The problem is that process_file also asks for user input. Is there are way to read from STDIN until EOF, store that, and then start asking for input again?

Solution 1

New answer 2021-10-18

bash: creating list from argument AND/OR standard input

Consider this little script:

#!/bin/bash

declare -a ARGS=("$@")

while read -rt .02 arg;do
    ARGS+=($arg)
    done

declare -p ARGS

You could call them argsAndInput.sh for sample, then ...

seq 1 10 | ./argsAndInput.sh foo bar baz
declare -a ARGS=([0]="foo" [1]="bar" [2]="baz" [3]="1" [4]="2" [5]="3" [6]="4" [
7]="5" [8]="6" [9]="7" [10]="8" [11]="9" [12]="10")

seq 1 10 | ./argsAndInput.sh 
declare -a ARGS=([0]="1" [1]="2" [2]="3" [3]="4" [4]="5" [5]="6" [6]="7" [7]="8"
 [8]="9" [9]="10")

./argsAndInput.sh foo bar baz 
declare -a ARGS=([0]="foo" [1]="bar" [2]="baz")

./argsAndInput.sh 
declare -a ARGS=()

Depending on your need, you could reduce timeout (read -t .02) or use:

while read -t 0 &&
    read -r arg;do
        ARGS+=($arg)
done

If you're sure your input will be ready before your bash should run.

Old answer

You may use xargs to transform STDIN (even with a lot of entries, widely bigger than command line buffer) to arguments.

Or else, maybe something like:

if [ $# -gt 0 ] ;then
    for filename in  "$@" ; do
        process_file "$filename"
    done
  else
    IFS=$'\n' read -d '' -r -a filenames
    for filename in "${filenames[@]}"; do
        process_file "$filename"
    done
  fi

or both:

  IFS=$'\n' read -d '' -r -a filenames
  for filename in "${filenames[@]}" "$@" ; do
        process_file "$filename"
    done
  fi

or by adding such an option, like -l stand for line arguments only, to ensure no read (wait) from STDIN:

case "$1" in
    -l )
        shift
        for filename in "$@" ; do
            process_file "$filename"
        done
        ;;
     * )
        for filename in "${filenames[@]}" "$@" ; do
            process_file "$filename"
        done
        ;;
    esac

(Not tested!)

Solution 2

Text processing tools traditionally read input from standard input when you don't specify any file name on the command line. You can check whether there are command line arguments by testing the $# variable. Assuming that process_file reads from standard input if you don't pass it an argument:

if [ $# -eq 0 ]; then
  process_file
else
  for x; do
    process_file "$x"
  done
fi

Alternatively, if process_file requires an argument, try passing it /dev/stdin to read from standard input:

if [ $# -eq 0 ]; then set /dev/stdin; fi
for x; do
  process_file "$x"
done

You asked to read a list of file names from standard input when there are no command line arguments. That's possible, of course, but I don't advise it, because it's unusual. Your users will have to learn a convention that's different from most other tools. Nonetheless, here's one way to do it:

if [ $# -eq 0 ]; then
  set -f # turn off globbing
  IFS='
' # set newlines to be the only field separator
  set -- $(cat)
  set +f; unset IFS
fi
for x; do
  process_file "$x"
done

Solution 3

This is a script I wrote to open files in both cygwin X11 gvim and windows gvim safely. It's designed as a shell wrapper and this is the part that does what you want:

if [ ${#} -ne 0 ]; then #if we have filenames as CLArgs...
    [[ ! -z ${DEBUG} ]] && echo "Got arguments [${#}]:'${@}'"
    for OFILE in "${@}"; do
        [[ -h ${OFILE} ]] && OFILE="$(readlink -f "${OFILE}")"
        [[ ${WINVIM} == true ]] && OFILE=$(cygpath -wal "${OFILE}")
        echo "\"${VIMRUN}\" --servername GVIM $RT \"${OFILE}\""
        "${VIMRUN}" --servername GVIM $RT "${OFILE}" &
        RT="--remote-tab"
    done
else #otherwise read from stdin safely and handle spaces...
    while read OFILE; do
        [[ -h ${OFILE} ]] && OFILE="$(readlink -f "${OFILE}")"
        [[ ${WINVIM} == true ]] && OFILE=$(cygpath -wal "${OFILE}")
        echo "\"${VIMRUN}\" --servername GVIM $RT \"${OFILE}\""
        "${VIMRUN}" --servername GVIM $RT "${OFILE}" &
        RT="--remote-tab"
    done
fi

Solution 4

This works for me:

for filename in ${*:-`cat`}; do
    process_file "$filename"
done

---

${*:-`cat`}

will expand first the command line arguments. If this is not defined (bash parameter substitution :- ), it will use stdin from the pipe.

Solution 5

With bash4.4 and above:

if [ "$#" -eq 0 ]; then
  readarray -d '' args
  set -- "${args[@]}"
fi

for arg do
   something with "$arg"
done

View more solutions

Comments

[Admin]

What's your question?

[Admin]

Did you mean: you want to make a script able to accept filenames either from command line and/or STDIN ?

[Admin]

Well I need it to be able to do either, but it doesn't need to do both at once. (Title edited)

[rahmu]

"That's possible, of course, but I don't advise it, because it's unusual." This is important. It's important to code in a way that is predictable for your users and potential future maintainers. Also, @Gilles, a small nit-pick, but your first sentence lacks a proper verb ;)

[Jeff]

I didn't know the <code>$# -eq 0</code> part, thanks! I've realized my problem was more comp.licated, but I'll mark this right.

[F. Hauri]

I was first! :-(

[Jeff]

Sorry, you were first.

[F. Hauri]

Just edited my answer, maybe could you find my idea interesting!