prevent xargs from quitting on error

Question

According to the man page, xargs will quit if one of the execution lines exits with an error of 255:

If any invocation of the command exits with a status of 255, xargs will stop immediately without reading any further input. An error message is issued on stderr when this happens.

How can I get xargs to not do this?

I have a 1500 or so line batch job that I want to run, 50 lines at a time. I was finding that it was always dying at a certain line, and not completing the job. Not good!

An even better question, the question describing what I am trying to do, is:

How can I run a 1500 line batch script, 50 lines at a time, so that it does not quit the job in the middle, and so that the output is captured to a log file of some kind?

Solution 1

You could wrap the perl script with another simple bash script:

#!/bin/bash
real-command "$@" || exit 0

This will call real-command passing it all the parameters that you pass to this fake-command and it will always return a 0 exit code (that means it is always successful) and xargs will never stop with this.

Solution 2

Similar to larsks answer but more explicit:

xargs sh -c "somecommand || true"

Solution 3

Just found a fun answer to this one, though its usefulness will depend on the command you're trying to run.

If you're using xargs to basically assemble a list of commands, you can get this behavior by telling xargs to echo the command, then piping to bash.

For example, if you're trying to delete a list of things that may or may not exist:

# presume this will fail in a similar way to your command
cat things_to_delete | xargs -n1 delete_command_that_might_exit

# instead echo the commands and pipe to bash
cat things_to_delete | xargs -n1 echo delete_command_that_might_exit | bash

This works because, first, xargs is only ever calling echo, so it won't see any errors. Then second, because bash's default behavior to continue execution after a failed statement.

To be more specific about my case, I was using this to remove a bunch of old application versions from AWS ElasticBeanstalk like so:

aws elasticbeanstalk describe-application-versions --application-name myapp |\
jq -r '.ApplicationVersions | sort_by(.DateCreated) | .[0:-10] | .[].VersionLabel' |\
xargs -n1 \
  echo aws elasticbeanstalk delete-application-version \
       --delete-source-bundle --application-name myapp --version-label |\
bash

Solution 4

You could write your xargs invocation to mask the return codes of your command lines. With something like the following,xargs will never see exit codes return by somecommand:

xargs sh -c "somecommand || :"

Solution 5

Following construction works for me:

ls | xargs -I % svn upgrade %

Even if svn upgrade failed on some element, process was continued

View more solutions

Comments

[JDS]

I've come up with a good solution: make sure the commands being processed do not exit with a 255 status! Additional Details The command being processed is a Perl script. The Perl die() function was being used in several places to exit out if some critical error occurred (e.g. could not connect to a database). However, die() always exits with error status 255. The solution in this case was to replace die() with a combination of print and exit(), along with a more reasonable error code ("1" worked in this case).

[JDS]

howdy. i could use that but the -exec option doesn't parallelize operations the way using xargs can and does

[Roger Dahl]

Thank you -- I didn't know that xargs could run commands in parallel. Cool. If you only want to minimize the number of command invocations, -exec has a + parameter.

[hukeping]

using sh -c wors for me!

[Iain Samuel McLean Elder]

I'll avoid it because piping to bash looks elegant until you try to cancel with CTRL+C. I had to cancel once for every single command that was remaining to be processed. Is there something I'm missing to be able to cancel at the xargs level rather than the bash level?

[matschaffer]

I suspect you'd need to kill the top level process (probably the terminal shell) that started the pipe for that.

[Wilfred Hughes]

If you're looking to run a single command for each line, you'll need -i too. This should work: xargs -i sh -c "somecommand {} || true"