Most efficient method to empty the contents of a file

Question

I am aware of three methods to delete all entries from a file.

They are

• >filename
• touch filename¹
• filename < /dev/null

Of these three I abuse >filename the most as that requires the least number of keystrokes.

However, I would like to know which is the most efficient of the three (if there are any more efficient methods) with respect to large log files and small files.

Also, how does the three codes operate and delete the contents?

---

¹Edit: as discussed in this answer, this actually does not clear the file!

Solution 1

Actually, the second form touch filename doesn't delete anything from the file - it only creates an empty file if one did not exist, or updates the last-modified date of an existing file.

And the third filename < /dev/null tries to run filename with /dev/null as input.

cp /dev/null filename works.

As for efficient, the most efficient would be truncate -s 0 filename (see here).

Otherwise, cp /dev/null filename or > filename are both fine. They both open and then close the file, using the truncate-on-open setting. cp also opens /dev/null, so that makes it marginally slower.

On the other hand, truncate would likely be slower than > filename when run from a script since running the truncate command requires the system to open the executable, load it, and then run it.

Solution 2

Other option could be:

echo -n > filename

From the man page of echo:

-n Do not print the trailing newline character.

Solution 3

There is a builtin command ":", which is available in sh,csh,bash and others maybe, which can be easily used with the redirecting output operator > truncate a file:

#!/usr/bin/env bash
:> filename

What I like on this is, that it does not need any external commands like "echo" etc.

One big advantage of truncating files instead of deleate/recreate them is, that running applications which works with this file (e.g. someone makes an tail -f filename or a monitoring software, ...) don't have to reopen it. They just can continue using the filedescriptor and gets all the new data.

Comments

[Robert Strauch]

What about truncate -s 0 filename?

[Stéphane Chazelas]

Very similar to Difference between cat and '>' to zero out a file where you'll find more information.

[Marco Marsala]

The first will work only if called from bash command line, but won't work if executed in a .sh file

[hbogert]

touch does not delete contents, but does change access time on the file. It does create an empty file if none existed.

[Stéphane Chazelas]

So why do you say that truncate is the most efficient?

[ash]

The truncate operation uses the ftruncate() or truncate() system call which does not bother to open the file. It also avoids the close() system call that cp and > filename methods need to call.

[Stéphane Chazelas]

Actually, it (at least the GNU one) does an open+ftruncate+close (in addition to the many system calls it does to load and initialise itself), as anyway, it would have to create the file if it didn't exist and truncate(2) doesn't do that.

[The Greeny]

If we use touch filename, will the inode remain same (provided there was a file before)?

[terdon]

@pMan yes, you can try it and check with ls -i

[javaPlease42]

I get this error truncate: not found.. What is the next best option since I cannot add the truncate function to this environment.

[Nathan Basanese]

// , Doesn't cp /dev/null filename lose the file ownership and permissions?

[ash]

/dev/null is owned by root and has very open file permissions, so that's probably a good thing. It will use the default settings (based on umask) when creating the new file.

[Sridhar Sarnobat]

The nice thing about truncate is that you can do it on multiple files in a single command without complicating things with xargs etc

[aalaap]

I feel >filename is the most efficient, because its faster to type. truncate may be technically faster, but it will never really be faster than > because you'll have to type out truncate -s 0 filename.

[ash]

@aalaap - if we're talking about speed of typing out commands, then I agree with you, that's probably about as fast as you can get. When talking about efficiency of the operation itself, such as when trying to keep down the overhead of truncating very rapidly (such as > 100 times per second), then the answer is very different.

[aalaap]

@ash Yes, I understand that. I was talking about a practical, real-world comparison between the two.

[Dani_l]

@aalaap your comment is not necessarily practical. If I have to truncate often, I would just function it to a short alias, and the keystroke count would be similar, so not an issue. However, if I do have to truncate very often, as on the order of 10k files, the issue of keystrokes is negligible compared to runtime.

[Polyphil]

How can I set the size? Say if I want 30000 null characters?

[ash]

There are shell settings, and more advanced syntax, that will prevent overwriting existing files. Look for the NO_CLOBBER option. set -o clobber will enable the file to be clobbered. Also, >| appears to override the noclobber setting

[Haxiel]

man bash describes the : shell builtin as having no effect.

[Mirko Steiner]

Yes, and you redirect this with > in to the file, which creates the file if it does not exists, and if it exists you truncate it to zero. Better said: you use the : to do nothing, and use > to redirect nothing to a file, and truncate it.

[terdon]

Why would you do that? > file is enough to truncate a file. You don't need any command, just the redirection operator.

[leleonp]

sometimes, > filename won't work. for example, in zsh. but : > filename works still.

[Mirko Steiner]

Bash and sh seems to like > myfile but e.g. csh errors with: Invalid null command.