Piping commands after a piped xargs

Question

HP-UX ***** B.11.23 U ia64 **** unlimited-user license

find . -type d -name *log* | xargs ls -la

gives me the directory names (the ones which contain log in the directory name) followed by all files within that directory.

The directories  /var/opt/SID/application_a/log/,  /var/opt/SID/application_b/log/,  /var/opt/SID/application_c/log/ and so on contain log files.

I want only the two latest logfiles to be listed by the ls command, which I usually find using ls -latr | tail -2.

The output has to be something like this..

/var/opt/SID/application_a/log/
-rw-rw-rw-   1 user1    user1      59698 Jun 11  2013 log1
-rw-rw-rw-   1 user1    user1      59698 Jun 10  2013 log2
/var/opt/SID/application_b/log/
-rw-rw-rw-   1 user1    user1      59698 Jun 11  2013 log1
-rw-rw-rw-   1 user1    user1      59698 Jun 10  2013 log2
/var/opt/SID/application_c/log/
-rw-rw-rw-   1 user1    user1      59698 Jun 11  2013 log1
-rw-rw-rw-   1 user1    user1      59698 Jun 10  2013 log2

find . -type d -name *log* | xargs ls -la | tail -2 does not give me the above result. What I get is a list of last two files of find . -type d -name *log* | xargs ls -la command.

So can I pipe commands after a piped xargs? How else do I query, to get the resultant list of files in the above format?

find . -type d -name *log* | xargs sh -c "ls -ltr | tail -10"

gives me a list of ten directory names inside the current directory which happens to be /var/opt/SID and that is also not what I want.

Solution 1

You are almost there. In your last command, you can use -I to do the ls correctly

-I replace-str

Replace occurrences of replace-str in the initial-arguments with names read from standard input.  Also, unquoted blanks do not terminate input items; instead the separator is the newline character.  Implies -x and -L 1.

So, with

find . -type d -name "*log*" | xargs -I {} sh -c "echo {}; ls -la {} | tail -2"

you will echo the dir found by find, then do the ls | tail on it.

Solution 2

Just in addition to fredtantini and as a general clarification (since the docs are a bit confusing):

The xargs -I {} will take the '{}' characters from the standard input and replace them with whatever comes in from the pipe. This means you could actually replace {} with any character combination (maybe to better suite your preferred programming flavor). For example: xargs -I % sh -c "echo %". If you always use the xargs -I {} you can replace it with xargs -i as it is the shorthand. EDIT: The xargs -i option has been deprecated, so stick to the xargs -I{}.

The sh -c will tell your bash/shell to read the next command from a string and not from the standard input. So writing sh -c "echo something" is equivalent to echo something.

The xargs -I {} sh -c "echo {}" will read the input you created with sh -c which is echo {}. Since you told it to replace {} with the arguments you got from the pipe, that's what will happen.

You can easily test this even without piping, just type the above command in a terminal. Whatever you write next will get outputted to the terminal (Ctrl-D to exit).

In the ls -la {} command the same thing happens again. The {} is replaced with the contents of the pre-pipe command.

Solution 3

GNU Parallel makes this kind of tasks easy:

find . -type d -name "*log*" | parallel --tag "ls -la {} | tail -2"

If you do not want to do a full install of GNU Parallel you can do a minimal installation: http://git.savannah.gnu.org/cgit/parallel.git/tree/README

Comments

[Admin]

You should quote the *log* otherwise the shell will expand it.

[Admin]

Be aware that sh -c expects the command name (parameter 0) as its second argument, so you should always do find . -type d -name *log* | xargs sh -c "ls -ltr | tail -10" lstail (notice the lstail at the end, which will serve as $0 for the created shell). Otherwise the first of your results will fill that role and go unused.

[PaulCrp]

I do with xargs -n 1 sh -c 'echo $0'

[mpuncel]

on macOS the replacement didn't work within the quoted command, so echo $0 was helpful (and more understandable)

[Benedikt Köppel]

To make it work with filenames with spaces, use -print0, xargs -0 and escape {} into double quotes inside the sh command: find . -type d -name "*log*" -print0 | xargs -0 -I {} sh -c "echo \"{}\";ls -la \"{}\" | tail -2"

[god]

I use "xargs -n 1 sh -c "echo \$0 | $func"` if I need to use the $func variable

[kelvin]

"If you always use the xargs -I {} you can replace it with xargs -i as it is the shorthand." Note that -i is not POSIX.