Can a TCP checksum fail to detect an error? If yes, how is this dealt with?

Question

If a TCP payload gets corrupted in transit the recomputed checksum won't match the transmitted checksum. Great, all fine so far.

If a TCP checksum gets corrupted in transit the recomputed checksum won't match the now corrupted checksum. Great, all fine so far.

What happens when both the payload and checksum get corrupted and the recomputed checksum, whilst different to what it should be, just happens to match the now corrupted checksum?

I can see with a good checksum algorithm (and additional checksums at lower levels) this might be very, very unlikely but isn't TCP meant to be 100% reliable? How does it resolve these false positives?

Solution 1

Something that should be noted here, and that most people overlook completely, is the fact, that the TCP checksum is actually a very poor checksum.

The TCP checksum is a 16-bit ones-complement sum of the data. This sum will catch any burst error of 15 bits or less, and all 16-bit burst errors except for those which replace one 1’s complement zero with another (i.e., 16 adjacent 1 bits replaced by 16 zero bits, or vice-versa). Over uniformly distributed data, it is expected to detect other types of errors at a rate proportional to 1 in 2^16. The checksum also has a major limitation: the sum of a set of 16-bit values is the same, regardless of the order in which the values appear.

Source: ftp://ftp.cis.upenn.edu/pub/mbgreen/papers/ton98.pdf

So if you randomly flip any number bits anywhere in the data part of the packet, the chances are 1 to 65536 that this error is not detected, even if you don't touch the checksum at all, as the new data, even though totally corrupt, has in fact the same checksum as the old one. If you just swap two 16 bit values in the data part, regardless which ones and regardless how often, the chances are even 100% that this error is not detected, since the order in which the 16 bit values appear in the data part of the packet is totally irrelevant to the value of the calculated checksum.

What I'm trying to say here is that you don't have to worry too much about the rather unlikely case that data and checksum both get corrupted and this error is not detected because the corrupted checksum matches the corrupted data, the truth is that every day millions of TCP packets on the Internet have only the data corrupted and this error is not detected because the uncorrupted checksum still matches the corrupted data.

If you need to transfer data and you want to be sure the data didn't get corrupted, the TCP checksum alone is certainly not enough for this task. I would even dare to say that a CRC checksum is not enough for this task, since a CRC32 may not detect an error where more than 32 bits in a row are affected (these errors can "cancel out" each other). The minimum checksum you'd need for ensuring flawless data transfer is the MD5 value of the data. Of course anything better than that (SHA-1, SHA-256, SHA-384, SHA-512, Whirlpool, and so on) will work even better, yet MD5 is sufficient. MD5 may not be secure enough for cryptographic security any longer (since it has been broken multiple times in the past), but as a data checksum MD5 is still absolutely sufficient.

Solution 2

No it can't be 100% reliable: this paper mentions 1 in 16 million to 10 billion packets not caught by the error control system. I'll let you calculate the occurences per day/week :)

Solution 3

Can a TCP checksum produce a false positive?

Yes. The checksum is considerably smaller than the packet, so many different packets can match a given checksum.

If yes, how is this dealt with?

In TCP, not at all. However, most data corruptions will be noticeable at a higher level, e.g. your XML is no longer well-formed; your email is no longer English, etc.

Solution 4

and additional checksums at lower levels

Some of these are stricter than checksums, e.g. Ethernet uses a CRC instead of a checksum.

this might be very, very unlikely but isn't TCP meant to be 100% reliable? How does it resolve these false positives?

I don't think it can. Even if it sent a duplicate via hard copy and carrier pigeon, a cosmic ray or quantum effects might theoreticaly mangle the duplicate too in exactly the same way. It's just very, very unlikely.

You can also implement arbitrarily strong integrity chcking at the application layer (above TCP), e.g. using cryptographic signing.

Solution 5

Assume

packet payload: 1000 byte

packet checksum: 2 byte

probability of packet with double error, one of wchich in checksum (assume P very small, less than 1/10^5):

A = 8P*(1000*8P) = 6*10^4 * P^2

probability of exact checksum:

B = 1/2^16 = 6/10^4

probability of false positive:

A * B = 40 * P^2 

The probability is low (P=1/10^6, then the probability of false positive A*B=4/10^11) but in any case with any crc algorithm it can't be zero. The probability of a random 1000 byte packet to appear as another random 1000 byte packet is P^8000, as if all bytes contain errors.

If P is high, for example from 1/10^3 to 1, the calculations above does not apply. In that case A=1 (all packets contain double errors) and the probability of false positive is just A*B = 6/10^4. It's not a very relevant case because more than 99% of received packets will contain errors in crc.

View more solutions