How to write a script that accepts input from a file or from stdin?

Question

How can one write a script that accept input from either a filename argument or from stdin?

for instance, you could use less this way. one can execute less filename and equivalently cat filename | less.

is there an easy "out of the box" way to do so? or do i need to re-invent the wheel and write a bit of logic in the script?

Solution 1

If the file argument is the first argument to your script, test that there is an argument ($1) and that it is a file. Else read input from stdin -

So your script could contain something like this:

#!/bin/bash
[ $# -ge 1 -a -f "$1" ] && input="$1" || input="-"
cat $input

e.g. then you can call the script like

./myscript.sh filename

or

who | ./myscript.sh

Edit Some explanation of the script:

[ $# -ge 1 -a -f "$1" ] - If at least one command line argument ($# -ge 1) AND (-a operator) the first argument is a file (-f tests if "$1" is a file) then the test result is true.

&& is the shell logical AND operator. If test is true, then assign input="$1" and cat $input will output the file.

|| is the shell logical OR operator. If the test is false, then commands following || are parsed. input is assigned to "-". The command cat - reads from the keyboard.

Summarising, if the script argument is provided and it is a file, then variable input is assigned to the file name. If there is no valid argument then cat reads from the keyboard.

Solution 2

read reads from standard input. Redirecting it from file ( ./script <someinput ) or through pipe (dosomething | ./script) will not make it work differently.

All you have to do is to loop through all the lines in input (and it doesn't differ from iterating over the lines in file).

(sample code, processes only one line)

#!/bin/bash

read var
echo $var

Will echo first line of your standard input (either through < or |).

Solution 3

You don't mention what shell you plan on using, so I'll assume bash, though these are pretty standard things across shells.

File Arguments

Arguments can be accessed via the variables $1-$n ($0 returns the command used to run the program). Say I have a script that just cats out n number of files with a delimiter between them:

#!/usr/bin/env bash
#
# Parameters:
#    1:   string delimiter between arguments 2-n
#    2-n: file(s) to cat out
for arg in ${@:2} # $@ is the array of arguments, ${@:2} slices it starting at 2.
do
   cat $arg
   echo $1
done

In this case, we are passing a file name to cat. However, if you wanted to transform the data in the file (without explicitly writing and rewriting it), you could also store the file contents in a variable:

file_contents=$(cat $filename)
[...do some stuff...]
echo $file_contents >> $new_filename

Read from stdin

As far as reading from stdin, most shells have a pretty standard read builtin, though there are differences in how prompts are specified (at the very least).

The Bash builtins man page has a pretty concise explanation of read, but I prefer the Bash Hackers page.

Simply:

read var_name

Multiple Variables

To set multiple variables, just provide multiple parameter names to read:

read var1 var2 var3

read will then place one word from stdin into each variable, dumping all remaining words into the last variable.

λ read var1 var2 var3
thing1 thing2 thing3 thing4 thing5
λ echo $var1; echo $var2; echo $var3
thing1
thing2
thing3 thing4 thing5

If fewer words are entered than variables, the leftover variables will be empty (even if previously set):

λ read var1 var2 var3
thing1 thing2
λ echo $var1; echo $var2; echo $var3
thing1
thing2
# Empty line

Prompts

I use -p flag often for a prompt:

read -p "Enter filename: " filename

Note: ZSH and KSH (and perhaps others) use a different syntax for prompts:

read "filename?Enter filename: " # Everything following the '?' is the prompt

Default Values

This isn't really a read trick, but I use it a lot in conjunction with read. For example:

read -p "Y/[N]: " reply
reply=${reply:-N}

Basically, if the variable (reply) exists, return itself, but if is's empty, return the following parameter ("N").

Solution 4

The simplest way is to redirect stdin yourself:

if [ "$1" ] ; then exec < "$1" ; fi

Or if you prefer the more terse form:

test "$1" && exec < "$1"

Now the rest of your script can just read from stdin. Of course you can do similarly with more advanced option parsing rather than hard-coding the position of the filename as "$1".

Solution 5

use (or chain off of) something else that already behaves this way, and use "$@"

let's say i want to write a tool that will replace runs of spaces in text with tabs

tr is the most obvious way to do this, but it only accepts stdin, so we have to chain off of cat:

$ cat entab1.sh
#!/bin/sh

cat "$@"|tr -s ' ' '\t'
$ cat entab1.sh|./entab1.sh
#!/bin/sh

cat     "$@"|tr -s      '       '       '\t'
$ ./entab1.sh entab1.sh
#!/bin/sh

cat     "$@"|tr -s      '       '       '\t'
$ 

for an example where the tool being used already behaves this way, we could reimplement this with sed instead:

$ cat entab2.sh
#!/bin/sh

sed -r 's/ +/\t/g' "$@"
$ 

View more solutions

Comments

[tvdo]

@PlasmaPower As long as the question is on-topic on SU, there is no requirement to ask on a different SE site. A lot of SE sites have overlap; generally we do not need to suggest an overlapping site unless the question is either off-topic (in which case, vote to migrate) or on-topic but not getting much of a response (in which case, the asker should flag for moderator-attention/migration, not cross-post).

[gilad hoch]

thanks! i choose the other answer because it suited me better. i was wrapping another script, and i didn't wanted to loop until all input recieved (could be a lot of input... would be wasteful).

[Suzana]

exec will try to execute the argument as a command which is not what we want here.

[Samin yeasir]

@Suzana_K: Not when it has no arguments, like here. In that case it just replaces file descriptors for the shell itself rather than a child process.

[Suzana]

I copied if [ "$1" ] ; then exec < "$1" ; fi in a test script and it gives an error message because the command is unkown. Same with the terse form.

[Suzana]

GNU bash 4.3.11 on Linux Mint Qiana

[Samin yeasir]

@Suzana_K: I just tested with bash 4.3.33 and it works fine. I suspect you have a typo somewhere in the script or else some weird configuration getting processed. Are you entering the command interactively or in a script you're executing? Perhaps you could pastebin the script if you'd like me to look at it (or better yet, post a question here about why it's not working despite that it should).

[Suzana]

Yes, that's weird. I'd also like to know the reason. I posted the output and the script to this gist.

[cmo]

what does && input="$1" || input="-" do and why is it outside the test operator?

[suspectus]

I've added an edit with some explanation which I hope will help.

[fregante]

This is fantastic! I'm using uglifyjs < /dev/stdin and it works wonderfully!

[yyny]

@Suzana_K This error occurs because exec is trying to open the file hallo, which does not exist. If you want to write to the file, flip the < to >.

[Admin]

What if the script has multiple arguments ($@)?

[baz]

@R.. Thanks. But exec < somefile doesn't make the shell read from the file by reading from stdin. Instead, it treats somefile as a shell script and executes it like source the script file. See superuser.com/questions/747884/…

[Samin yeasir]

@Tim: That's false. I don't know where you got that misconception but it's easily testable as well as clear from the specification.

[baz]

My typo. See unix.stackexchange.com/q/447317/674

[Samin yeasir]

@Tim: That's in an interactive shell that's reading commands from stdin, as opposed to a shell executing a script.

[baz]

Bash manual doesn't say exec < somefile behaves differently in a noninteractive shell from an interactive shell. From what reference did you get it?

[Samin yeasir]

It's not that exec < somefile behaves any differently. It doesn't. It's that the shell's basic command processing behaves differently in interactive mode (reading commands from stdin) vs executing a script (reading commands from the script).