Argument list too long for xargs/exec
Solution 1
You have multiple mistakes. You should escape the *
globbing. You have to put {}
between quotes (for filename security), and you have to end the -exec
with \;
.
find ./ -iname out.\* -type f -exec mv "{}" /home/user/trash \;
find -name ./paramsFile.\* -exec cat "{}" >> parameters.txt \;
The problem here is that *
is matching all the files in your directory, thus giving you the error. If find
locates the files instead of shell globbing, xargs
gets individual filenames that it can use to construct lines of the correct length.
Solution 2
Try this:
find . -iname 'out.*' -type f -exec mv '{}' /home/user/trash \;
find . -name 'paramsFile.*' -print0 | xargs -0 cat >> parameters.txt
The >>
is to make sure multiple invocations of cat
(if you really have a huge number of files) output to the same file, without overwriting the result from previous calls. Also, make sure parameters.txt
starts out empty (or delete it first).
Related videos on Youtube
ohblahitsme
Updated on September 18, 2022Comments
-
ohblahitsme over 1 year
I'm working on a CentOS server and I have to move around and cat together millions of files. I've tried many incarnations of something like the below, but all of them fail with an argument list too long error.
command:
find ./ -iname out.* -type f -exec mv {} /home/user/trash find ./paramsFile.* -exec cat > parameters.txt
error:
-bash: /usr/bin/find: Argument list too long -bash: /bin/cat: Argument list too long
or
echo ./out.* | xargs -I '{}' mv /home/user/trash (echo ./paramsFile.* | xargs cat) > parameters.txt
error:
xargs: argument line too long xargs: argument line too long
The second command also never finished. I've heard some things about globbing, but I'm not sure I understand it completely. Any hints or suggestions are welcome!
-
ohblahitsme almost 11 yearsThanks for the answer! It seems to still be too long for find and cat.
-
jjlin almost 11 yearsOh, I guess I didn't really read that carefully. I updated the answer with the other call. Also, +1 to @Bernhard for bringing up some good points that I forgot about.
-
Martin Dorey over 3 yearsIt's great that the reason for the >> is given, but it's wrong. The redirection happens once, in the parent shell, regardless of how many times cat is invoked by xargs.