Network Traffic Log

Question

Background - On my "home" network I have a Linksys WTR45GL router providing my internet access as well as a wireless AP.

Connected I have * 2 Windows PCs (wired) * At least one laptop (Wired) * Some 802.11 enabled handheld consoles (PSPs) * A Nintendo Wii * Some windows XP pcs used by the people in the granny flat.

Where I live, South Africa, well, 1Gb worth of monthly cap is, while not expensive, costly enough that I'd like to be sure that all the bandwidth used by devices on my network is ... well ... legitimate and not the result of neighbors parasiting my wireless, malware or just the result of "liberal" download policies in my software.

I got the Linksys WRT45GL on the understanding that there were custom firmwares (DD-WRT and Tomato) that allowed bandwidth tracking, but there doesn't seem to be any facility to get a log of traffic that can be examined to see (a) which local devices were the biggest consumers of bandwidth and (b) what they were connected to.

What tools are there for logging traffic such that, when it gets to that OMG moment in the month when all my bandwidth is gone, I have a chance to find out what used it all up (and hopefully attempt some corrective action).

---

Solutions I have tried:

1. MRTG - Installation of MRTG is complicated. It needs to install on a server PC - i.e. one running a web server. It's also "limited" to doing SNMP monitoring - which means it can only report on aggregate traffic per interface. Yay - I can confirm that there is Gb of traffic traversing my router. I already knew that!

2. PRTG's free version is too limited to do anything useful with. Or the 'sensors' its suports are too limited. In addition to SNMP trapping it does packet sniffing, but still the capability to drill down in its reports is lacking. At least it can break down traffic a bit by protocol. So now I know that 80% of my traffic is HTTP. Still no idea what the sources and destinations are of the HTTP traffic. Also entirely unclear if its capable of measuring traffic over a not-well-know-port.

3. This Wiki Page details how to (a) configure DD-WRT to broadcast traffic using RFlow and MACudp to a server on the network. (b) The server needs to be running RFlowCollector which captures the data, allows limited examination of the data, and (c) puts it in a mySQL database - where it can then be analysed with 3rd party tools - or (d) just MSQL Query Analyzer.

Except for the fact that RFlowCollector is a badly written windows GUI app with no option to install as a service: Meaning that it can only capture data as long as I am physically logged in. Oh yes, and it uses 60% of my CPU even when theres noting being logged. Its own UI can't do any drill down analysis of the data (but does tease), so at the end of all this I'm left with a large mySQL database of 'raw' captured data to analyse.

4. Wireshark has also been suggested. As a network protocol analyser it certainly gives the low level details - as long as I'm on the correct network segment Wireshark can be used to sniff and produce a comprehensive list of all the tcp and udp traffic. Which is a problem. Wireshark is just too low level. Theres too much data.

Is it really this hard to try and do a simple analysis of a networks bandwidth? And by simple I mean, given a "chunk" of bandwidth usage on my router, ask something to tell me assign it to (a) local hosts, (b) remote hosts and (c) by protocol.

Solution 1

NetFlow is a protocol, available on DD-WRT "standard" distributions by enabling the "RFlow" service. The RFlow service can be pointed to a PC on the LAN running a Netflow logger or analysis tool.

The RFlow collector I mentioned in the original post is rubbish. There are however a number of free tools that can display Netflow data in realtime in a pretty GUI. Most are substantially limited in their free editions however :(

"free" Tools that I've found (all are limited feature 'free-as-in-beer' versions of a horribly expensive commpercial 'enterprise' product) include:

• Paessler's PRTG
• Plixer's Scrutinzer
• Solarwinds' Free Realtime NetFlow Analyzer

Solution 2

you can run arpwatch and mrtg in your dd-wrt router and that will tell you #1 when a new client associates with your AP and issues a DHCP request (thru arpwatch) and #2 how much bandwidth you have consumed (thru mrtg). the total bandwidth addon to mrtg is at bjorn.swift.is/traffic/

Comments

[Traveling Tech Guy]

Doesn't the router itself provide a packet log?

[Chris Becke]

It provides a log, but not a packet level log. Home routers just don't have the memory to log that amount of data, which is why rFlow is used to offset traffic logging to another device.

[Chris Becke]

maybe im doing it wrong, but MRTG doesn't tell me anything I don't all ready know. I already know there is a lot of traffic through my router. Is there a way I can't see that breaks the bandwidth down per local host so I can pinpoint those of my devices that are responsible for the bulk of my traffic?