how to sum output of awk or other expression with xargs

Question

Suppose i have the following bash shell script:

#!/bin/bash
export  count=0;
for i in `ls ./mydoc` ;do

     pdfinfo ./mydoc/$i | egrep Pages |awk {'print $2'} |xargs -+ $count   ;
                                                        ^^^^^^^^^^^^^^^^^
done;

echo $count;

It enumarates pages of each pdf files, but i don't know how to sum them together. !!!!!

Solution 1

Just do it in the awk, forget xargs.

awk '{total += $2} END {print total}'

but you need tto run awk only once, with all of the loop's output piped into it. And why not eliminate the separate grep too...

for i in mydoc/* ; do
    pdfinfo $i
done | awk '/^Pages/ { total += $2 } END { print total }'

Solution 2

Try the built-in calculation method:

#!/bin/bash
count=0
for i in ./mydoc/* ; do
  count=$(( $count + $(pdfinfo "$i" | grep Pages |awk {'print $2'}) ))
done
echo $count

Solution 3

The accepted awk answer by Alan is good, but here's a generic solution using xargs and bc. The idea is to generate a list of numbers in some way, use xargs to join them all on one line separated by spaces, and then use sed to change the spaces to + characters (tr would work too). pipe that into bc.

The same method can be used to construct a regexp from a list of strings/regexps, just change the spaces to | (extended regexp) or \| (basic regexp) instead of +:

for i in mydoc/* ; do pdfinfo $i ; done | \
  awk '/^Pages/ {print $2}' | xargs | sed -e 's/ /+/g' | bc

NOTE: if there are many thousands of numbers generated, exceeding the shell's command line length limit, xargs may generate multiple lines. Since the output of bc qualifies as "generate a list of numbers in some way", the solution is to pipe the output of bc into xargs | sed -e 's/ /+/g' | bc again.

for i in mydoc/* ; do pdfinfo $i ; done | \
  awk '/^Pages/ {print $2}' | xargs | sed -e 's/ /+/g' | bc | \
  xargs | sed -e 's/ /+/g' | bc

xargs | sed -e 's/ /+/g' | bc | xargs | sed -e 's/ /+/g' | bc can, of course, be put into a shell script, function, or alias.

and here's an example of constructing a regexp using this method. If search.txt contains foo, bar, baz, quux (one word per line) then:

$ cat search.txt | xargs | sed -e 's/ /|/g'
foo|bar|baz|quux

the useless-use-of-cat is a place-holder for this example - replace with any pipeline that generates a list of words or regexp patterns.

If any of the search patterns contain space characters, you'll have to change them to something else (choose something not likely to be in the input) temporarily, before piping into xargs and then change them back again after the sed. e.g. if the 'bar' line of search.txt has a trailing space:

$ cat search.txt | sed -e 's/ /XXX_SPACE_CHARACTER_XXX/g' | xargs | sed -e 's/ /|/g' -e 's/XXX_SPACE_CHARACTER_XXX/ /g'
foo|bar |baz|quux

Comments

[Alan Curry]

@jofel I didn't notice how broken the rest of the script was. Updated with a rewrite.

[PersianGulf]

well done, When you migrate completely to awk , everything ws OK.

[PersianGulf]

it's very nice and clean.