Measure traffic from apache access log

Question

Is there any way to measure how much traffic there were used in one Apache log file?

Format:

66.249.72.214 - - [05/Nov/2011:12:47:37 +0200] "GET /produktas/565638 HTTP/1.1" 200 4699 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

How I understand 4699 are bytes that were transferred excluding headers.

I need a simple solution (maybe a little bash script) to sum bytes in every log's line.

Solution 1

Try this. I tested it on a local file but cannot tell if it works on all configurations/locales/...

cat apache.log | perl -e 'my $sum=0; while(<>) { my ($traffic) = m/\[.+\] ".+" \d+ (\d+)/; $sum += $traffic}; print "$sum\n"'

Update Jan 2017: Meanwhile I've learned some more Perl and that's how I'd do it today:

cat apache.log | perl -nE '/\[.+\] ".+" \d+ (\d+)/; $sum += $1; END {say $sum}'

Solution 2

For detailed log file monitoring and actual bandwidth usage, go for AWStats.

It takes the Apache log file as input and give you very detailed analysis of the visitors and bandwidth, with graphs.

You can also try GoAccess.

Solution 3

Apache Access Log — Global bandwidth usage :

awk '{ s += $10 } END { print "Total ", s/1024/1024 " Mo", "- Moyenne ", s/NR/1024/1024 " Mo", "- Accès ", NR }' access.log

And for a file :

grep NAME_OF_RESOURCE_HERE /var/log/apache2/access.log* | awk '{ s += $10 } END { print "Total ", s/1024/1024 " Mo", "- Moyenne ", s/NR/1024/1024 " Mo", "- Accès ", NR }'

You get something like this : Total 301.985 Mo - Moyenne 0.0430055 Mo - Accès 7022

Solution 4

I think you need to use apachetop utility, try to install from APT with next command:

sudo apt-get install apachetop

And then run it with command:

sudo apachetop -f /path/to/access.log

And you are rock! :)

Solution 5

We needed to get the traffic of last X days. I’m really not into perl, so what i did is:

zcat $(find -name yourvhost_access.log*.gz -mtime -3 2>/dev/null| xargs ) \ 
| awk '$10 ~ /^[0-9]+$/ {print $10}' \
| paste -sd+ \
| bc 

Steps:

1. find last 3 achived accesslogs
2. print position 10 if its a Number - payload should be here
3. put it together with "+"
4. calculate

View more solutions

Comments

[Justinas Lelys]

Thank you! I have modified it a little bit to show in gigabytes: [code]cat access.log | perl -e 'my $sum=0; while(<>) { my ($traffic) = m/[.+] ".+" \d+ (\d+)/; $sum += $traffic}; printf("%.3f/1024/1024/1024); print " GB\n"'[/code]