xargs - append each argument with a parameter

Question

I know that, given l="a b c",

echo $l | xargs ls

yields

ls a b c

Which construct yields

mycommand -f a -f b -f c

Solution 1

One way to do it:

echo "a b c" | xargs printf -- '-f %s\n' | xargs mycommand

This assumes a, b, and c don't contain blanks, newlines, quotes or backslashes. :)

With GNU findutil you can handle the general case, but it's slightly more complicated:

echo -n "a|b|c" | tr \| \\0 | xargs -0 printf -- '-f\0%s\0' | xargs -0 mycommand

You can replace the | separator with some other character, that doesn't appear in a, b, or c.

Edit: As @MichaelMol notes, with a very long list of arguments there is a risk of overflowing the maximum length of arguments that can be passed to mycommand. When that happens, the last xargs will split the list and run another copy of mycommand, and there is a risk of it leaving an unterminated -f. If you worry about that situation you could replace the last xargs -0 above by something like this:

... | xargs -x -0 mycommand

This won't solve the problem, but it would abort running mycommand when the list of arguments gets too long.

Solution 2

A better way to address it (IMO) would be:

• in zsh:

  l=(a b c)
  mycommand -f$^l
  

  or using array zipping so the argument be not attached to the option:

  l=(a b c) o=(-f)
  mycommand "${o:^^l}"
  

  That way, it still works if the l array contains empty elements or elements containing spaces or any other problematic character for xargs. Example:

  $ l=(a '' '"' 'x y' c) o=(-f)
  $ printf '<%s>\n' "${o:^^l}"
  <-f>
  <a>
  <-f>
  <>
  <-f>
  <">
  <-f>
  <x y>
  <-f>
  <c>
  

• in rc:

  l=(a b c)
  mycommand -f$l
  

• in fish:

  set l a b c
  mycommand -f$l
  

(AFAIK, rc and fish have no array zipping)

With old-style Bourne-like shells like bash, you could always do (still allowing any character in the elements of the $@ array):

set -- a b c
for i do set -- "$@" -f "$i"; shift; done
mycommand "$@"

Solution 3

The simplest way to prefix arguments is via printf in conjunction with command substitution:

l="a b c"

mycommand $(printf ' -f %s' $l)

Alternatively, the command substitution $() can be rewritten by piping to xargs:

printf ' -f %s' $l | xargs mycommand

The command substitution allows to control location of the dynamic arguments in the argument list. For instance, you can prepend, append, or even place the arguments anywhere in between any other fixed arguments to be passed to mycommand.

The xargs approach works best to append arguments to the end, but it requires a more obscure syntax to handle different placement of dynamic arguments among fixed ones.

Comments

[Michael Mol]

You run a pretty ugly risk of exceeding ARG_MAX and having a -f separated from its paired parameter.

[Satō Katsura]

@MichaelMol That's a good point, but I don't think there is any meaningful way to handle that situation without knowing more about mycommand. You could always add -x to the last xargs.

[Michael Mol]

I think the proper solution is probably not to use xargs at all, and just use find if it can be used. This solution is dangerous; you should at least warn of the failure case in your answer.

[Satō Katsura]

@MichaelMol I really don't see how find would be a better general solution, particularly when the initial arguments aren't filenames. :)

[Michael Mol]

We don't know what the initial arguments are; we only see the example given, not the scenario that inspired the question. Intuition suggests that with an argument named -f, and with an example tool ls used for illustration, @not-a-user is dealing with filenames. And given find offers the -exec argument, which allows you to construct a command-line, it's fine. (So long as mycommand is permitted to execute more than once. If it's not, then we have another problem with the use of xargs here...)

[Satō Katsura]

@MichaelMol As I said: xargs -x.

[Michael Mol]

And now we're not processing all input lines. Are you going to reflect these caveats in your answer? edit: Ah, I see you did, now.

[MoonCheese62]

Instead of echo $l | xargs printf ... (which is closer to the actual question) one could write more simply printf ... $l.

[Peter Cordes]

Another way to do it in bash is with a named array variable. for i; do args+=('-f' "$i");done; mycommand "${args[@]}". IDK if this is faster, but appending 2 elements to an array seems like it should be O(n), while your set loop probably copies and re-parses the accumulated arg list every time (O(n^2)).

[Admin]

Instead of -x you can set -n to a reasonable even value like 1024 if multiple executions of mycommand make sense