How to compress and clean logs with logrotate but not rotate them

Question

I have a Tomcat server which create access logs via a valve (org.apache.catalina.valves.FastCommonAccessLogValve). This valve takes care of rotating the access log file, but not of compressing them or deleting them after some time.

For the moment, I have a cron job that uses find [...] -mtime +30 [...] to compress and delete the logs. I would rather use logrotate, so that log rotation is in a centralized place for all logs. I dont like having a seperate solution just for Tomcat.

I tried to read logrotate documentation, but I am still a bit lost. Can I use logrotate just to compress and cleanup log files ? How would I do that ?

Or turning the problem around, is there a Tomcat access log valve that will compress and cleanup log files ?

Thank for your help !

Solution 1

Fairly simple one that i have seen works like this.

Create a file in /etc/logrotate.d called tomcat containing the following:-

/var/log/tomcat/catalina.out { 
  copytruncate 
  daily 
  rotate 7 
  compress 
  missingok 
}

This runs daily, compresses the file, and keeps 7 days worth (rotate 7). copytruncate means that it'll copy then truncate the original file so that there is no need to restart tomcat. missing ok will not error if it's not there.

The access.log Valve can be changed to not rotate by adding rotatable=false:-

<Valve className="org.apache.catalina.valves.AccessLogValve
     ...
     ...
     suffix="log" rotatable="false" />

Solution 2

Modified TimP's script--Added deleting very old files, added scan for old compressed files.

#!/bin/bash
#
# TPP 2013-02-21
# RJK 2014-08-14
#
# Tomcat apps use a variety of loggers, mostly log4j.
# These rotate, conflicting with logrotate, the unix log rotation system.
#
# Some files eg catalina.out
# are rotated to a backup containing a date eg catalina.2013-01-06.log
# which can then be compressed with bz2 to catalina.2013-01-06.log.bz2
# or removed if older than a given number of days(MTIME).
#
cd /var/log/tomcat6
# 2013-02-21
DATE=`date --rfc-3339=date`
YEAR=`date +%Y`
MILLENIUM=20
# 2014-08-14
MTIME=14
# 2014-08-14
#for f in $(find catalina* |grep -v bz2 |grep -v '$DATE' |grep $YEAR)
for f in $(find catalina* |grep -v bz2 |grep -v '$DATE' |grep $MILLENIUM)
do
 # 2014-08-14
 if test `find $f -mtime +$MTIME`
 then
   echo "rm -f $f"
   rm -f $f
 else
   echo "bzip2 $f"
   bzip2 $f
 fi
done
# However others are active whilst containing a date
# so we will find all and not compress the most recent
for l in 'localhost*' 'opt-db*' 'opt*' 'host-manager*' 'manager*'
do
 export previous=
 for f in $(find $l |grep -v bz2 |sort)
 do
  if [ "${previous}" != "" ]
  then
    echo "bzip2 ${previous}"
    bzip2 $previous
  fi
  export previous=$f
 done
done
# 2014-08-14
for f in $(find *bz2)
do
 if test `find $f -mtime +$MTIME`
 then
   echo "rm -f $f"
   rm -f $f
 fi
done
exit 0 

Solution 3

It's surprisingly simple. Just tell logrotate which files you specifically want to rotate. nocreate tells logrotate not to recreate an empty file after moving the old one (if you're rotating files into a subfolder).

/var/log/tomcat/catalina.out.* { 
  daily 
  nocreate
  compress 
  missingok 
}

Solution 4

I did not wish to change the Tomcat configuration, so created a script which compressed the rotated files

#!/bin/bash
#
# TPP 2013-02-21
#
# Tomcat apps use a variety of loggers, mostly log4j.
# These rotate, conflicting with logrotate, the unix log rotation system. 
#
# Some files eg catalina.out
# are rotated to a backup containing a date eg catalina.2013-01-06.log
# which can then be compressed with bz2 to catalina.2013-01-06.log.bz2
#

cd /var/log/tomcat6

# 2013-02-21
DATE=`date --rfc-3339=date`
YEAR=`date +%Y`

for f in $(find catalina* |grep -v bz2 |grep -v '$DATE' |grep $YEAR)
do
 echo "bzip2 $f" 
 bzip2 $f
done

# However others are active whilst containing a date
# so we will find all and not compress the most recent
for l in 'localhost*' 'opt-db*' 'opt*' 'host-manager*' 'manager*'
do
 export previous=
 for f in $(find $l |grep -v bz2 |sort)
 do
  if [ "${previous}" != "" ]
  then
    echo "bzip2 ${previous}" 
    bzip2 $previous
  fi
  export previous=$f
 done
done

exit 0

View more solutions

Comments

[Janning]

BTW: FastCommonAccessLogValve is deprecated in 6.0 tomcat.apache.org/tomcat-6.0-doc/api/org/apache/catalina/val‌​ves/…

[Guillaume]

My problem is that unlike catalina.out, my access log is already rotated.

[Decado]

Okay, access log rotation can be disabled by setting rotatable=false in the Valve arguments. Answeraltered to include this.

[Bill McGonigle]

domain is gone ... Wayback: web.archive.org/web/20170410025123/http://www.techiewiki.org‌​/…