Is there a simple logging based web proxy?

Question

For the fun, I'm curious to save all web traffic I send/receive. Is there any FLOSS web proxy software that runs on Ubuntu Linux that will just save, to a file, the headers, url, datetime and the content of all HTTP traffic that moves through it? If there isn't, I'll just write my own, but no point re-inventing the wheel.

UPDATE The reason I want this is just cause I'm a bit of a data hoarder. And I'm wondering is there is some sort of way to record all my web traffic.

Solution 1

Squid will do this quite happily. Installation should be no more than apt-get install squid, and the config should be fairly trivial too.

Solution 2

You can do this with tcpdump too. It's a bit agressive , but it can capture all the traffic sent or received from a webserver (port 80/tcp)

tcpdump 'src port 80 or dst port 80'

From client side and using firefox, you can use the slogger extension, that dumps all data sent or received by firefox in differnt formats

Solution 3

squid mentioned by Tom O'Connor will be probably best solution. but if you want completly passive tool that can work on the mirroring port [ in opposite to the squit that works 'in-line' ] - take a look at httpry - specialized sniffer for analyzing http traffic. it's small and quite flexible.

Comments

[Araejay]

Squid will do the proxying fine, but it's the logging and 'record everything' that I'm looking for.

[Araejay]

Yes I had used slogger, and that's the sort of thing I want to have. I was thinking that a http proxy was 'the right way to do it'.

[Admin]

Can't downvote this, but to best of my knowledge Tinyproxy can't log HTTP request bodies or headers out of the box. It's just URLs that are logged. Or I'm really missing something.

[Admin]

I tried this. Tinyproxy doesn't log the bodies.