run commands inside awk before print

Question

I wish to list directories while calculating their size and print the whole thing in a simple command, using ls and awk, work great to control the list and what I wish to display :

ls -AlhF /usr | awk '{print $6, $7, $8, "\011", $9 }'

But I would like to go a bit further and to get the size of the folders or the files, use $9 with the command du -s with something like this :

ls -AlhF /usr | awk '{print $6, $7, $8, "\011", $9, du -s /usr/$9 }'

I've tried many ways to write it, with backticks, doublequote, but I always end up with errors or unwanted results.

Solution 1

To answer the question and ignoring the issues about parsing the output of ls for now, to get the output (stdout) of a command in awk, you do:

cmd="that command"
output=""
while ((cmd | getline line) > 0)
  output = output line RS
status = close(cmd)

How the exit status is encoded in the status variable varies from one awk implementation to the next. All you can rely on is that it will be 0 if and only if cmd succeeds.

getline starts a shell (usually a POSIX shell, but on some systems, that could be a Bourne shell) to parse that command line. So it's important to properly quote data in there. Best is to use single quotes which is the safest.

Here since the output will be a single line anyway (your approach can't handle file names with any spacing characters, let alone newlines), you only need to do one getline:

 awk -v q="'" '
   function shquote(s) {
     gsub(q, q "\\" q q, s)
     return q s q
   }
   {
     cmd = "du -sk " shquote("/usr/" $9)
     cmd | getline du_output
     status = close(cmd)
   }'

If you call getline without a variable name, then it sets $0 which can make it easier to extract the disk usage:

DIR=/usr export DIR
LC_ALL=C ls -l -- "$DIR" | awk -v q="'" '
  function shquote(s) {
    gsub(q, q "\\" q q, s)
    return q s q
  }
  {
    date = $6 " " $7 " " $8
    name = $9
    cmd = "du -sk " shquote(ENVIRON["DIR"] (ENVIRON["DIR"] ~ /^\/*$/ ? "" : "/") name)
    cmd | getline
    disk_usage = $1
    print date "\t" name "\t" disk_usage
  }'

Solution 2

ls -AlhF /usr | awk '{print "echo "$6" "$7" "$8" \011 "$9" $(du -s /usr/"$9")" }' | sh

Construct an echo and du command with awk and then execute it through piping to sh.

Comments

[Alessio]

Don't Parse ls. Also, awk is not sh. backticks and $() aren't supported. Use the system() function instead.

[chistof]

Thanks for the suggestions, I will think about not parsing ls after, but for now, after trying and searching, I got something almost working : ls -lAhF /usr | awk '{system("du -s /usr/"$9" | cut -f1")}{print $9}' The problem is that both information don't appear on the same line...

[Raman Sailopal]

Don't use system as it is prone to the risk of command injection. It is best to pipe through to sh first. See me solution

[Stéphane Chazelas]

That qx(du -s "$dir/$_") is a command injection vulnerability. You could use the environ: $ENV{FILE}="$dir/$_" and qx'du -sk -- "$FILE"', or use open to also avoid having to run a shell.

[Alessio]

yeah, i was being lazy. easily fixed. I'll open a pipe to read from du instead. I thought of using Filesys::DiskUsage, but wanted to avoid uncommon modules. (by "uncommon", i mean "not packaged for debian" :)

[Stéphane Chazelas]

The stripping of trailing / is not valid for /path/to/script /

[Alessio]

du -s uses tab as separator on my system, not a single space. du (GNU coreutils) 8.26 on debian sid.