find + xargs: argument line too long

Question

I have a line like the following:

find /foo/bar -name '*.mp4' -print0 | xargs -i {} -0 mv -t /some/path {}

but I got the following error:

xargs: argument line too long

I am confused. Isn't the use of xargs supposed to precisely help with this problem?

Note: I know that I can techincally use -exec in find, but I would like to understand why the above fails, since my understanding is that xargs is supposed to know how to split the input into a manageable size to the argument that it runs. Is this not true?

This is all with zsh.

Solution 1

Well for one thing the -i switch is deprecated:

-i[replace-str]
     This  option  is a synonym for -Ireplace-str if replace-str is specified. 
     If the replace-str argument is missing, the effect is the same as -I{}. 
     This option is deprecated; use -I instead.

So when I changed your command around to this, it worked:

$ find /foo/bar -name '*.mp4' -print0 | xargs -I{} -0 mv -t /some/path {}

Example

$ find . -print0 | xargs -I{} -0 echo {}
.
./.sshmenu
./The GIT version control system.html
./.vim_SO
./.vim_SO/README.txt
./.vim_SO/.git
./.vim_SO/.git/objects
./.vim_SO/.git/objects/pack
./.vim_SO/.git/objects/pack/pack-42dbf7fe4a9b431a51da817ebf58cf69f5b7117b.idx
./.vim_SO/.git/objects/pack/pack-42dbf7fe4a9b431a51da817ebf58cf69f5b7117b.pack
./.vim_SO/.git/objects/info
./.vim_SO/.git/refs
./.vim_SO/.git/refs/tags
...

Use of -I{}

This approach shouldn't be used since running this command construct:

$ find -print0 ... | xargs -I{} -0 ...

implicitly turns on these switches to xargs, -x and -L 1. The -L 1 configures xargs so that it's calling the commands you want it to run the files through in a single fashion.

So this defeats the purpose of using xargs here since if you give it 1000 files it's going to run the mv command 1000 times.

So which approach should I use then?

You can do it using xargs like this:

$ find /foot/bar/ -name '*.mp4' -print0 | xargs -0 mv -t /some/path

Or just have find do it all:

$ find /foot/bar/ -name '*.mp4' -exec mv -t /some/path {} +

Solution 2

The option -i takes an optional argument. Since you put a space after -i, there was no argument to the -i option and therefore the subsequent -0 was not an option to xargs but the second of 6 operands {} -0 mv -t /some/path {}.

With only the option -i, xargs expected a newline-separated list of file names. Since there was probably no newline in the input, xargs received what looked like a huge file name (with embedded null bytes, but xargs didn't check that). This single string containing the whole output of find was longer than the maximum command line length, hence the error “command line too long”.

Your command would have worked with -i{} instead of -i {}. Alternatively, you could have used -I {}: -I is similar to -i, but takes a mandatory argument, so the next argument passed to the xargs is used as the argument of the -I option. Then the argument after that is -0 which is interpreted as an option, and so on.

However, you shouldn't use -I {} at all. Using -I has three effects:

• -I turns off quote processing, which -0 already does.
• -I changes the string to replace, but {} is the default value.
• -I causes the command to be executed separately for each input record, which is useless here since your command (mv -t) is specifically intended to cope with multiple files per invocation.

Either drop -I and -i altogether

find /foo/bar -name '*.mp4' -print0 | xargs -0 mv -t /some/path {}

or drop xargs and use -exec:

find /foo/bar -name '*.mp4' -exec mv -t /some/path {} +

Comments

[Amelio Vazquez-Reina]

Thanks! When you said "This approach shouldn't be used" which approach should be used instead then? Would "find /foot/bar/ -name '*.csv' -print0 | xargs -0 mv -t some_dir'" be a better solution? If so, how does xargs know in this case where in the mv command to feed in the arguments it gets from the pipe? (does it always place them last?)

[slm]

@user815423426 - Doing it with just the find ... -exec ... is a better way or if you want to use xargs the find ... | xargs ... mv -t ... is fine too. Yup it always puts them last. That's why that method needs the -t.