How to parse TCP packet payload

Question

I'm using pcap to capture TCP packets for which I would like to parse the payload. My strategy is as follows:

1. Get the ethernet header and check if it has type ETHERTYPE_IP (IP packet)
2. Check if the IP packet has protocol IPPROTO_TCP (TCP packet)

3. Check for payload size > 0 (size = ntohs(ip_header->total_length - ip->header_length*4 - sizeof(struct tcp_header)).

4. parse payload (grab the host url)

I haven't begun parsing the payload yet because I am getting discrepancies. Below is a printout of the payload of 10 captured TCP packets, using filter "host = www.google.com".

packet number: 3 : TCP Packet: Source Port: 80 Dest Port: 58723 No Data in packet

packet number: 4 : TCP Packet: Source Port: 58723 Dest Port: 80 No Data in packet

packet number: 5 : TCP Packet: Source Port: 58723 Dest Port: 80 Payload : GET / HTTP/1.1 Host: www.google.com User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_5; en-us) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4 Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,/;q=0.5 Accept-Language: en-us Accept-Encoding: gzip, deflate Cookie: THICNT=25; SID=DQAAAKIAAAB2ktMrEftADifGm05WkZmlHQsiy1Z2v- Connection: keep-alive

packet number: 6 : TCP Packet: Source Port: 80 Dest Port: 58723 No Data in packet

packet number: 7 : TCP Packet: Source Port: 80 Dest Port: 58723 Payload: \272نu\243\255\375\375}\336H\221\227\206\312~\322\317N\236\255A\343#\226\370֤\245[\327`\306ըnE\263\204\313\356\3268 )p\344\301_Y\255\267\240\222x\364

packet number: 8 : TCP Packet: Source Port: 58723 Dest Port: 80 No Data in packet

packet number: 9 : TCP Packet: Source Port: 80 Dest Port: 58723 Payload: HTTP/1.1 200 OK Date: Mon, 29 Nov 2010 10:11:36 GMT Expires: -1 Cache-Control: private, max-age=0 Content-Type: text/html; charset=UTF-8 Content-Encoding: gzip Server: gws Content-Length: 8806 X-XSS-Protection: 1; mode=block \213

Why is there a discrepancy in the payloads and the ports? Ideally I would like to only parse packets like packet 5. How do I ignore packets like 7 and 9?

Solution 1

Only by analyzing content. Nothing in IP or TCP header what can mark "HTTP Request" packets. Even "first data packet in connection" wouldnot work because there are persistent connections.

Also, to be completely sure about catching all URIs you need to reassemble TCP stream and parse HTTP request: URI can be split on two or more packets.

Solution 2

Like the IP header, the TCP header is variable-length as well. You are not taking that into account. Rather than blindly subtracting sizeof(struct tcp_header)) from the total packet size, you need to locate the TCP header within the IP data, then use its length field (which needs to be multiplied by 4, just like the IP header length field does) to know where the actual data payload is located.

Solution 3

Your size calculation is incorrect - you can't do the subtraction in network-host-order, you have to convert each field to host-byte-order first:

size = ntohs(ip_header->total_length) - ntohs(ip->header_length) * 4 - sizeof(struct tcp_header))

However, as Remy Lebeau points out, you actually need to examine the offset field in the TCP header to know where the payload starts.

The difference between packet 5 and packet 7 is that the former is going from the client, to the server, and the latter is a response from the server to the client. This is why the ports are switched around - the source and destination addresses will be switched also.

If you want to only look at packets coming from the client, check that the source address is equal to the client's address.