Piping output to text file within a for loop
Solution 1
There really is no need for reproducing find
's output ina shell loop. If you want to pack the list of filenames, the generic formula is:
find ... | gzip > logfile.gz
If you want to gzip the files themselves, it changes to:
find ... | tar -czvf archive.tar.gz -T -
which tells tar
to read list of file names to work on from a file and the single -
stands for standard input. (The -T
AKA --files-from=
option is present in GNU tar, I'm not sure about other flavours.) Of course this breaks if you manage to work on files which contain \n
in their names.
Solution 2
You cannot post-process the output of find reliably. Use -exec
in find
:
find . -type f ! -name '*.gz' ! -name '*.Z' -mtime +7 -exec sh -c '
for i do
printf "%s\n" "Will be compressing file $i"
gzip "$i"
done' sh {} + >> log
With the GNU implementation of find
, you can even get away without running sh:
find . -type f ! -name '*.gz' ! -name '*.Z' -mtime +7 \
-printf 'Will be compressing file %p\n' -exec gzip {} + >> log
Solution 3
put your redirect (>>
) outside the loop (after done
) like so:
for i in $(find . -type f \( ! -name '*.gz' \) -a \( ! -name '*.Z' \) -mtime +7);
do echo "Will be compressing file ${i}";
##gzip $i;
done >> $LOGCLEAN_FILE
Or, if you plan on eventually uncommenting your gzip
command you might consider:
for i in $(find . -type f \( ! -name '*.gz' \) -a \( ! -name '*.Z' \) -mtime +7);
do gzip "$i" && echo "Successfully compressed ${i}";
done >> $LOGCLEAN_FILE
the &&
means only do the next command if the previous command exits with 0
(no error).
Solution 4
As filenames can have both spaces and new-lines a probable approach would be:
#!/bin/bash
logclean_file="logclean_file.txt"
ts="$(date)"
printf "%s\n" "$ts" > "$logclean_file"
# Set IFS blank
# -r Backslash does not act as an escape character. The backslash
# is considered to be part of the line. In particular, a
# back-slash-newline pair may not be used as a line continuation.
# -d delim
# The first character of delim is used to terminate the input
# line, rather than newline.
#
# Here setting -d to nul or 0x00. This enables us to capture any file-
# names with the print0 from find.
#
# fn The variable to read into.
#
while IFS= read -r -d $'\x00' fn; do
printf "Will perhaps be compressing file %s\n" "$fn" >> "$logclean_file"
# If file + gz does not exist
if [[ ! -e "$fn.gz" ]]; then
if gzip command "$fn"; then
echo "Horray! success!" >> "$logclean_file"
else
echo "Harf! Gzip failed." >> "$logclean_file"
fi
else
echo "Nah. Already exists." >> "$logclean_file"
fi
done < <(find . -type f \( ! -name '*.gz' \) -a \( ! -name '*.Z' \) -print0)
# Notice -print0 at end which means find will print filenames, - separating
# them with 0x00 instead of new-line
About portability:
Neither echo
nor print
are really portable. While print
is unique to ksh93, echo
only got standardized very late in the POSIX process, and older versions cannot be relied on to produce predictable results. See echo vs print and Why is printf better ....
printf "Some %s\n" "$var" >> "$foo"
# Or
echo "Some $var" >> "$foo"
About Style:
- quote variables.
- Consider using lowercase of user variables (your own variables).
Some ref:
For the TLDP guides, and if you are using stylish, I'd recommend one of these for screen-reading.
Related videos on Youtube
user3412206
Updated on September 18, 2022Comments
-
user3412206 almost 2 years
I'm trying to do the following within a
for
loop:- Find files that satisfy a condition
- Echo the name of the files to a log file.
- Gzip the file.
I can get the script to find the files and echo their names to the screen but cannot pipe them to a file. (I have not got as far testing the gzip part)
Here's the relevant portion of the script (The
$LOGCLEAN_FILE
exists and is written to in an earlier portion of the script):for F in `find . -type f \( ! -name '*.gz' \) -a \( ! -name '*.Z' \) -mtime +7` do { print "Will be compressing file ${F}" >> $LOGCLEAN_FILE } ; ##gzip $F done
If I run the script without the
" >> $LOGCLEAN_FILE"
text then the output displays on the screen.What am I doing wrong?
Criticisms are welcome - I'm still learning.
-
vonbrand over 11 yearsThe command to write output in the shells is
echo
, notprint
. -
Stéphane Chazelas over 11 years@vonbrand, it is
print
in the shell that morgon specified (ksh
). -
Gilles 'SO- stop being evil' over 11 yearsWhat is the value of
$LOGCLEAN_FILE
? If it's a relative path, have you changed directories? -
user3412206 over 11 yearsGilles - your comment actually pointed me to the problem with my script. The script changed directory after the setup of the $LOGCLEAN_FILE variable and wrote the rest of the output to a different directory instead of the intended location. If you can submit the comment as an answer I'll mark it as the answer.
-
Stéphane Chazelas over 11 yearsNobody mentioned
bash
.print
is a ksh (and zsh) command. -
Gilles 'SO- stop being evil' over 11 yearsGood advice, but I'm puzzled: why isn't the script as posted working (assuming tame file names)?
-
user3412206 over 11 yearsWill test once I am back at my desk. Thanks for the feedback
-
user3412206 over 11 yearsMy problem with using the -exec option is that it changes the result of the previous find statement. If I run the find command as type in my post at a shell prompt , then it displays a correct list of files, If I add "-exec ls -al {} \;" then the list displayed is markedly different. I suspect however that I will be submitting a separate question about that.
-
user3412206 over 11 yearsOn what basis do you make the comment? You cannot post-process the output of find reliably. I use simliar commands to move files, mount file systems etc regularly in daily work.
-
Henk Langeveld over 11 years@Sukminder - thanks for Greycat's links on shell quotes. It's a good resource.
-
user3412206 over 11 yearsThanks to all for the advice will check out sukminder's link references
-
Runium over 11 years@HenkLangeveld: Thanks for edit! I won't delete after all then.
-
ThorSummoner over 9 yearsThis does not address piping output inside the for loop!