Count requests from access log for the last 7 days

Question

I would like to parse an access log file and have returned the amount of requests for the last 7 days. I have this command

cut -d'"' -f3 /var/log/apache/access.log | cut -d' ' -f2 | sort | uniq -c | sort -rg

Unfortunately, this command returns the amount of requests since the creation of the file and sorts it into HTTP-code categories. I would like just a number, no categories, and only for the last 7 days.

Solution

I'd set up log rotation daily (how to do this would be dependent on your OS), then use the same command above on the 7 most recent logs. As for your existing log, either use a tool like grep to extract just the days you want, or split that log into logs for each day.

If you want something more elegant than that, I'd just look for one of the myriad log parsing tools already out there.

Here's an example to split up your existing log: Split access.log file by dates using command line tools

Comments

[Hedam]

My logs are rotated by size - not time. I will take a look at the link.

[Hedam]

Your link is basically about months. I would like to split into weeks.

[Hedam]

Does that work on CentOS?

[Admin]

The answer marked accepted only breaks it down to months. If you scroll down there are plenty more examples. Every single other example on there now breaks it down into individual days. From there it's up to you how you define a week and combine the log(s) together if you require it by the week.

[Matthew Steeples]

Afraid not, the utility is Windows only. It's a relatively old app so you may be able to get it working with Wine. Have never personally tried though.

[Hedam]

How to be exact?

[Admin]

Exact about what? Pick out the example that makes the most sense to you, generate the files, then combine 7 days worth cat day1 day2 day3 day4 day5 day6 day7 > week1, then run your original command on that.

[Hedam]

I see. I can't seem to create my own version. I'm not that good at bash, pearl etc.

[Admin]

Copy the example starting with awk, paste that somewhere in a text editor. Change the very end that says 'access-log' with 'access.log' (name taken from your example above). Save that file (plain text format!) with a name such as 'split_logs.sh'. Copy that file into the same directory your access.log file is and run 'sh split_logs.sh'. You should end up with a separate file for each day. Then you can run your original command against whatever files you want.

[Hedam]

It's for a dashboard I am making for my customers, so an External log system is not possible.

[Jack ilkgu]

If you decide to go down this route, you would install Splunk on servers you control, preferably on dedicated instances. I think Loggly only offered hosted solutions.