Understanding Bash's Read-a-File Command Substitution
Solution 1
The <
isn't directly an aspect of bash command substitution. It is a redirection operator (like a pipe), which some shells allow without a command (POSIX does not specify this behavior).
Perhaps it would be more clear with more spaces:
echo $( < $FILE )
this is effectively* the same as the more POSIX-safe
echo $( cat $FILE )
... which is also effectively*
echo $( cat < $FILE )
Let's start with that last version. This runs cat
with no arguments, which means it will read from standard input. $FILE
is redirected into standard input due to the <
, so cat
puts its contents are put into standard output. The $(command)
subsitution then pushes cat
's output into arguments for echo
.
In bash
(but not in the POSIX standard), you can use <
without a command. bash
(and zsh
and ksh
but not dash
) will interpret that as if cat <
, though without invoking a new subprocess. As this is native to the shell, it is faster than literally running the external command cat
. *This is why I say "effectively the same as."
Solution 2
$(<file)
(also works with `<file`
) is a special operator of the Korn shell copied by zsh
and bash
. It does look a lot like command substitution but it's not really.
In POSIX shells, a simple command is:
< file var1=value1 > file2 cmd 2> file3 args 3> file4
All parts are optional, you can have redirections only, command only, assignment only or combinations.
If there are redirections but no command, the redirections are performed (so a > file
would open and truncate file
), but then nothing happens. So
< file
Opens file
for reading, but then nothing happens as there's no command. So the file
is then closed and that's it. If $(< file)
was a simple command substitution, then it would expand to nothing.
In the POSIX specification, in $(script)
, if script
consists only of redirections, that produces unspecified results. That's to allow that special behaviour of the Korn shell.
In ksh (here tested with ksh93u+
), if the script consists of one and only one simple command (though comments are allowed before and after) that consists only of redirections (no command, no assignment) and if the first redirection is a stdin (fd 0) input only (<
, <<
or <<<
) redirection, so:
$(< file)
$(0< file)
-
$(<&3)
(also$(0>&3)
actually as that's in effect the same operator) $(< file > foo 2> $(whatever))
but not:
$(> foo < file)
- nor
$(0<> file)
- nor
$(< file; sleep 1)
- nor
$(< file; < file2)
then
- all but the first redirection are ignored (they are parsed away)
- and it expands to the content of the file/heredoc/herestring (or whatever can be read from the file descriptor if using things like
<&3
) minus the trailing newline characters.
as if using $(cat < file)
except that
- the reading is done internally by the shell and not by
cat
- no pipe nor extra process is involved
- as a consequence of the above, since the code inside is not run in a subshell, any modification remain thereafter (as in
$(<${file=foo.txt})
or$(<file$((++n)))
) - read errors (though not errors while opening files or duplicating file descriptors) are silently ignored.
In zsh
, it's the same except that that special behaviour is only triggered when there's only one file input redirection (<file
or 0< file
, no <&3
, <<<here
, < a < b
...)
However, except when emulating other shells, in:
< file
<&3
<<< here...
that is when there are only input redirections without commands, outside of command substitution, zsh
runs the $READNULLCMD
(a pager by default), and when there are both input and output redirections, the $NULLCMD
(cat
by default), so even if $(<&3)
is not recognized as that special operator, it will still work like in ksh
though by invoking a pager to do it (that pager acting like cat
since its stdout will be a pipe).
However while ksh
's $(< a < b)
would expand to the content of a
, in zsh
, it expands to the content of a
and b
(or just b
if the multios
option is disabled), $(< a > b)
would copy a
to b
and expand to nothing, etc.
bash
has a similar operator but with a few differences:
-
comments are allowed before but not after:
echo "$( # getting the content of file < file)"
works but:
echo "$(< file
# getting the content of file
)"
expands to nothing.
- like in
zsh
, only one file stdin redirection, though there's no fall back to a$READNULLCMD
, so$(<&3)
,$(< a < b)
do perform the redirections but expand to nothing. - for some reason, while
bash
does not invokecat
, it still forks a process that feeds the content of the file through a pipe making it much less of an optimisation than in other shells. It's in effect like a$(cat < file)
wherecat
would be a builtincat
. - as a consequence of the above, any change made within are lost afterwards (in the
$(<${file=foo.txt})
, mentioned above for instance, that$file
assignment is lost afterwards).
In bash
, IFS= read -rd '' var < file
(also works in zsh
) is a more effective way to read the content of a text file into a variable. It also has the benefit of preserving the trailing newline characters. See also $mapfile[file]
in zsh
(in the zsh/mapfile
module and only for regular files) which also works with binary files.
Note that the pdksh-based variants of ksh
have a few variations compared to ksh93. Of interest, in mksh
(one of those pdksh-derived shells), in
var=$(<<'EOF'
That's multi-line
test with *all* sorts of "special"
characters
EOF
)
is optimised in that the content of the here document (without the trailing newline characters) is expanded without a temporary file or pipe being used as is otherwise the case for here documents, which makes it an effective multi-line quoting syntax.
To be portable to all versions of ksh
, zsh
and bash
, best is to limit to only $(<file)
avoiding comments and bearing in mind that modifications to variables made within may or may not be preserved.
Solution 3
Because bash
does it internally for you, expanded the filename and cats the file to standard output, like if you were to do $(cat < filename)
. It's a bash feature, maybe you need to look into the bash
source code to know exactly how it works.
Here the the function to handle this feature (From bash
source code, file builtins/evalstring.c
):
/* Handle a $( < file ) command substitution. This expands the filename,
returning errors as appropriate, then just cats the file to the standard
output. */
static int
cat_file (r)
REDIRECT *r;
{
char *fn;
int fd, rval;
if (r->instruction != r_input_direction)
return -1;
/* Get the filename. */
if (posixly_correct && !interactive_shell)
disallow_filename_globbing++;
fn = redirection_expand (r->redirectee.filename);
if (posixly_correct && !interactive_shell)
disallow_filename_globbing--;
if (fn == 0)
{
redirection_error (r, AMBIGUOUS_REDIRECT);
return -1;
}
fd = open(fn, O_RDONLY);
if (fd < 0)
{
file_error (fn);
free (fn);
return -1;
}
rval = zcatfd (fd, 1, fn);
free (fn);
close (fd);
return (rval);
}
A note that $(<filename)
is not exactly equivalent to $(cat filename)
; the latter will fail if the filename starts with a dash -
.
$(<filename)
was originally from ksh
, and was added to bash
from Bash-2.02
.
Related videos on Youtube
Stanley Yu
Updated on September 18, 2022Comments
-
Stanley Yu almost 2 years
I am trying to understand how exactly Bash treats the following line:
$(< "$FILE")
According to the Bash man page, this is equivalent to:
$(cat "$FILE")
and I can follow the line of reasoning for this second line. Bash performs variable expansion on
$FILE
, enters command substitution, passes the value of$FILE
tocat
, cat outputs the contents of$FILE
to standard output, command substitution finishes by replacing the entire line with the standard output resulting from the command inside, and Bash attempts to execute it like a simple command.However, for the first line I mentioned above, I understand it as: Bash performs variable substitution on
$FILE
, Bash opens$FILE
for reading on standard input, somehow standard input is copied to standard output, command substitution finishes, and Bash attempts to execute the resulting standard output.Can someone please explain to me how the contents of
$FILE
goes from stdin to stdout? -
Stanley Yu over 9 yearsSo in the last paragraph when you say "
bash
will interpret that ascat filename
", do you mean this behavior is specific to command substitution? Because if I run< filename
by itself, bash does not cat it out. It will output nothing and return me back to a prompt. -
Adam Katz over 9 yearsA command is still needed. @cuonglm altered my original text from
cat < filename
tocat filename
which I oppose and may revert. -
Stanley Yu over 9 yearsThanks for all the answers so far, Adam. But if bash is interpreting
< filename
ascat < filename
in the context of command substitution, why does the man page say$(< file)
is faster? This statement, from the start, led me to believe cat was somehow redundant. This is clearly not the case if what you say is true. Or maybe I missed something. -
Adam Katz over 9 years@cuonglm, I have reverted your edit.
cat < -
does indeed fail, but-
also does not work with< -
so it is consistent. -
Adam Katz over 9 years@StanleyYu,
$(< file)
is faster because it doesn't spawn a new process (cat
).<
is a builtin. -
Stanley Yu over 9 yearsOh, I think I understand now. So it's basically doing the same thing as
cat < filename
but with the added benefit of staying within bash. Thanks! -
Gilles 'SO- stop being evil' over 9 yearsYour explanation (especially the second sentence) is wrong. No pipe is involved in
<$FILE
. A pipe is involved in the command substitution$(…)
only. -
Adam Katz over 9 years@Gilles: I thought the term "pipe" referred to any redirection operator rather than just
|
. I have rephrased given the verbiage in the man page fordash
. With that clarification, I do not believe that the term "pipe" (or "pipeline") is relevant w.r.t.$()
command substitution. -
Gilles 'SO- stop being evil' over 9 yearsA pipe is a type of file. The shell operator
|
creates a pipe between two subprocesses (or, with some shells, from a subprocess to the shell's standard input). The shell operator$(…)
creates a pipe from a subprocess to the shell itself (not to its standard input). The shell operator<
does not involve a pipe, it only opens a file and moves the file descriptor to standard input. -
Gilles 'SO- stop being evil' over 9 yearsOh, also bash doesn't actually launch
cat
, it does the job internally. -
Stanley Yu over 9 yearsGood stuff. Thanks for explaining the additional intricacies of the shell and confirming that bash does it internally, Gilles. And, yes, I understand that cat is not called. When Adam said "is the same as" I understood it to mean something more along the lines of "provides the same result as", not literally
<
=cat <
. -
cuonglm over 9 years@AdamKatz: No,
cat filename
will fail with filename starts with-
,cat < filename
doesn't. -
cuonglm over 9 years@AdamKatz: And also
<
is not a builtin in bash, it's redirection operator. -
cuonglm over 9 years@AdamKatz: I mean
cat <filename
in my first editing, maybe I was sleepy. -
Adam Katz over 9 years@cuonglm: Your edits are not helping. The fact that
cat -file
will fail whilecat < -file
will not is unrelated.<
is built into bash. While it's not a builtin command, it is also not an external command. I'll use the term "native" instead. -
cuonglm over 9 years@AdamKatz: That's why I said
<filename
is equivalent tocat < -filename
isntead ofcat -file
. Just want to insistcat $FILE
#<$FILE
. -
Stéphane Chazelas almost 7 years
< file
is not the same ascat < file
(except inzsh
where it's like$READNULLCMD < file
).< file
is perfectly POSIX and just opensfile
for reading and then does nothing (sofile
is close straight away). It's$(< file)
or`< file`
that is a special operator ofksh
,zsh
andbash
(and the behaviour is left unspecified in POSIX). See my answer for details. -
Tim almost 7 yearsIs it correct that
$(<)
is an operator on filenames? Is<
in$(<)
a redirection operator, or not an operator on its own, and must be part of the entire operator$(<)
? -
Scott - Слава Україні almost 7 yearsTo put @StéphaneChazelas’s comment in another light: to a first approximation,
$(cmd1) $(cmd2)
will typically be the same as$(cmd1; cmd2)
. But look at the case wherecmd2
is< file
. If we say$(cmd1; < file)
, the file is not read, but, with$(cmd1) $(< file)
, it is. So it is incorrect to say that$(< file)
is just an ordinary case of$(command)
with a command of< file
.$(< …)
is a special case of command substitution, and not a normal usage of redirection. -
Scott - Слава Україні almost 7 years@StéphaneChazelas: Fascinating, as usual; I’ve bookmarked this. So,
n<&m
andn>&m
do the same thing? I didn’t know that, but I guess it’s not too surprising. -
Scott - Слава Україні almost 7 yearsRight. Except, of course, the two-argument form of the file descriptor duplication call is called
dup2()
;dup()
takes only one argument and, likeopen()
, uses the lowest available file descriptor. (Today I learned that there is adup3()
function.) -
Stéphane Chazelas almost 7 years@Scott, yes sorry, see also Significance of arrows symbols in duplicating/closing file descriptors under bash
-
eel ghEEz over 5 yearsNot sure if Bash's process substitution
<(list)
is portable:$ cat <(echo test)
produces outputtest
. -
Stéphane Chazelas over 5 years@eelghEEz, that's another feature of ksh copied by zsh and bash (not pdksh). That's in effect the reverse: get a file out of string instead of a string out of a file.