Rotate a file that's open and being written at all times

Question

I have an linux application that continually writes logging information into a log file, eg. /var/log/application.log. As the application does not rotate the file automatically, this log file can reach a size of gigabytes in some weeks, so I want to be able to rotate this file properly

My main concern here is that to rotate a file that is opened by the application at all times, I will probably need to:

1. Move the file to its rotated form /var/log/application.log -> /var/log/application.log.2013-01-28

2. Create an empty /var/log/application.log. Obs: At this point the application process is still writing to /var/log/application.log.2013-01-28

3. Change the file descriptor of the application process to point back again to /var/log/application.log

So, am I right? If so, how can i do this? (mainly the changing the file descriptor part)

If i am not, what is the correct way and how to do it?

Solution 1

Write a logrotate config to use copytruncate

copytruncate
    Truncate the original log file in place after creating a copy, instead of moving the 
    old log file and optionally creating a new one. It can be used when some program
    cannot be told to close its logfile and thus might continue writing (appending) to
    the previous log file forever. Note that there is a very small time slice between
    copying the file and truncating it, so some logging data might be lost. When this
    option is used, the create option will have no effect, as the old log file stays
    in place.

Solution 2

Most such applications respond to a signal, such as SIGHUP, and will close and reopen their log files on receipt of the signal. Check your application's documentation for the correct signal to send.

Comments

[ewwhite]

Do you need to clear the file or just rotate it?

[David Schwartz]

What application? The application has to provide support to do this properly. (Otherwise, you can a very ugly thing -- attach to the process, open the new file, dup2 the new descriptor over the old one, then close the new descriptor.)

[Bruno Polaco]

Interesting aproach Schwartz. I am curious to see that in action and will play with it a while. Anyway, the application is in house and im looking for a more general solution. I like kormoc´s answer btw

[Bruno Polaco]

@ewwhite I need to rotate it, I cant afford to lose the log data

[lsd]

And if it doesn't, you can restart the application as a fallback. That is what syslog does usually, and quite a few other programs.

[Bruno Polaco]

Interesting, truncating the file after copying it seems quite clever, even with the data loss margin. I didnt notice this option on logrotate. Thx for the insight :)