Linux: Can I monitor all http connections to my site live? like tail -f /file/log.log only for http connections?

Question

I want to monitor whether or not people are accessing my site live. I want to see the http connections made to my site in terminal/ssh. Something like a tail -f /file/log.log function but I believe there is a way without using a log file right?

I was reading some of the other peoples' questions here, it seems "IPTstate" is what I'm looking for, anyone to confirm this does what I'm asking? I'm not much of a server admin.

Solution 1

There is no one answer.

• IPTables packet and byte counts will show new connections coming into to your server. I only use this to verify the connections are going through the right rules.
• I generally look at the access log for the web server. It provides a bunch of information you can't easily get from monitoring the tcp stack. tail -f on the log is useful to see if connections are getting processed properly.
• Watching netstat can show connections to port 80 and/or port 443. TCPDUMP will show packet level monitoring data.
• Enabling server-status in apache allows you to query what the server is doing and has recently done.
• A system monitor like munin can track accesses and load on the web server.
• A log analysis tool like awstats will give you relatively current data with history and a lot of useful breakdowns. This is useful to see trends over time.
• Google analytics will give you information similar to awstats, but based on active code run when your site is accessed.

Which solution is appropriate depends on your needs. You may choose to use more than one solution.

Solution 2

This will work but it's hacky...

watch -n 0.5 "netstat -ano | grep -v unix | grep ESTABLISHED | grep \:80"

Solution 3

apachetop could be a solution for you.

This is a console-based (non-gui) monitoring tool which reads the server-status pages from one or more Apache servers and combines the information onto one easy monitoring screen.

A Sample output would look like this:

last hit: 20:03:08         atop runtime:  0 days, 00:00:40             20:03:09
All:          203 reqs (   5.2/sec)       2678.4K (   68.7K/sec)      13.2K/req
2xx:     203 ( 100%) 3xx:       0 ( 0.0%) 4xx:     0 ( 0.0%) 5xx:     0 ( 0.0%)
R ( 30s):     148 reqs (   4.9/sec)       2286.0K (   76.2K/sec)      15.4K/req
2xx:     148 ( 100%) 3xx:       0 ( 0.0%) 4xx:     0 ( 0.0%) 5xx:     0 ( 0.0%)

 REQS REQ/S    KB KB/S URL
    2  0.10   7.9  0.4*/images/persons/ahmet-buke.jpg
    2  0.10  10.3  0.5 /images/persons/rm_24_s_0.gif
    2  0.14  86.0  6.1 /images/banners/kitap/hegel-14kasim-o(1).jpg
    2  0.09   2.7  0.1 /themes/yeni//includes/languages/turkish/images/buttons
    2  0.09  11.7  0.5 /images/services/pinterest.jpg
    2  0.07   5.8  0.2 /themes/yeni/images/kedi.gif
    2  0.09   4.4  0.2 /images/services/google-plus.jpg

Solution 4

TCPDUMP will be able to show you live connections to the server.

View more solutions

Comments

[Darius]

Hacky as in not a good way to do it, or makes me/my server vulnerable?

[Stefan Lasiewski]

Yes TCPDUMP can do this, but can you provide a quick example showing how tcpdump can do this? The requester asked for something like tail -f /file/log.log, so I assume he'd like to know what page was requested, header information, etc.

[Vick Vega]

Take a look at the following example please: danielmiessler.com/study/tcpdump

[egorgry]

hacky as in not the best solution.

[Scott Pack]

Can you elaborate any on what kind of information apachetop gives you, any specifics as to how it can be used for this specific situation?

[R T]

and how to list like trail as this one just refresh the output