Ls with spaces + variables

Question

I want to do something like this, but it doesn't save the variable after the piping ends:

fs=( )
echo ${fs[@]}
ls -A1 |
while read f
do
    echo ${fs[@]}
    fs+=( "$f" )
    echo ${fs[@]}
done
echo "All files/dirs: "${fs[@]}

With files 1, 2 and 3 in the current dir I get this output:

# Output:


1
1
1 2
1 2
1 2 3
All files/dirs: 

How do I keep the fs variable's value, after the piping ends?

Update:

• Can't use *, becuase i need hidden files
• Can't just use fs=($(ls)), while sometimes the file/dir names will have spaces in them

Solution

Don't parse the output of ls.

The way to list all the files in a directory in the shell is simply *.

for f in *; do …

In shells with array support (bash, ksh, zsh), you can directly assign the file list to an array variable: fs=(*)

This omits dot files, which goes against your use of ls -A. In bash, set the dotglob option first to include dot files, and set nullglob to have an empty array if the current directory is empty:

shopt -s dotglob nullglob
fs=(*)

In ksh, use FIGNORE=".?(.)"; fs=(~(N)*). In zsh, use the D and N glob qualifiers: fs=(*(DN)). In other shells, this is more difficult; your best bet is to include each of the patterns * (non-dot files), .[!.]* (single-dot files, not including . and double-dot files) and ..?* (double-dot files, not including .. itself), and check each for emptiness.

set -- *; [ -e "$1" ] || shift
set .[!.]* "$@"; [ -e "$1" ] || shift
set ..?* "$@"; [ -e "$1" ] || shift
for x; do …  # short for for x in "$@"; do …

---

I'd better explain what was going wrong in your attempt, too. The main problem is that each side of a pipe runs in a subprocess, so the assignments to fs in the loop are taking place in a subprocess and never passed on to the parent process. (This is the case in most shells, including bash; the two exceptions are ATT ksh and zsh, where the right-hand side of a pipeline runs in the parent shell.) You can observe this by launching an external subprocess and arranging for it to print its parent's process ID¹´²:

sh -c 'echo parent: $PPID'
{ sh -c 'echo left: $PPID >/dev/tty'; echo $? >/dev/null; } |
{ sh -c 'echo right: $PPID >/dev/tty'; echo $? >/dev/null; }

In addition, your code had two reliability problems:

• Don't parse the output of ls.
• read mangles whitespace and backslashes; to parse lines, you need while IFS= read -r line; do …

For those times when you do need to parse lines and use the result, put the whole data processing in a block.

producer … | {
  while IFS= read -r line; do
    …
  done
  consumer
}

¹ _{Note that $$ wouldn't show anything: it's the process ID of the main shell process, it doesn't change in subshells.}
² _{In some bash versions, if you just call sh on a side of the pipe, you might see the same process ID, because bash optimizes a call to an external process. The fluff with the braces and echo $? defeat this optimization.}

Comments

[Gilles 'SO- stop being evil']

`ls` will choke horribly on file names containing whitespace, unprintable characters, or shell wildcards. Don't do this.

[Michael Mrozek]

This doesn't appear to answer the question at all, if I'm understanding it correctly. In any case, you shouldn't post answers just because you lack the rep for a comment, and definitely don't tell people to go to SO for their completely on-topic questions

[Tyilo]

One last thing: if I normally wanted to use sudo ls -A1 to grab the files, how would I do that with the *?

[Tyilo]

I want to list the files in a partition's .Trashes folder (/Volumes/External HD/.Trashes/), which requires root priviledges.

[Gilles 'SO- stop being evil']

@Tyilo: Hmm, tricky. Bash doesn't support null characters in general, but sudo bash -c 'shopt -s nullglob dotglob; printf "%s\\0" *' | while IFS= read -r -d "" filename; do … seems to work (using a null character as the separator, this way all file names work).

[Angel Todorov]

The way to remove the pipeline and hence remove the subshell (and keep the variable in the current shell) is to use process substitution: while IFS= read -r -d "" filename; do f+=("$filename"); done < <(shopt -s nullglob dotglob; printf "%s\\0" *) -- this also keeps the shopt in a subshell.

[tcoolspy]

It is a potential answer but the asker rulled it out already based on needing more features than just globbing. @Michael's other points stand, don't post question clarification comments in answers and don't send people off-site for on-topic issues.

[Kedar Vaidya]

Thats all fine and valid, I just don't think it should've been deleted. Furthermore, I posted this answer -before- he updated his post.