TCP loopback connection vs Unix Domain Socket performance

Question

Working on an Android and iOS based application which require communication with a server running in the same device. Currently using TCP loopback connection for communicating with App and Server (App written in user layer, server written in C++ using Android NDK)

I was wondering if replacing inter communication with Unix Domain socket would improve the performance?

Or in-general is there any evidence/theory that proves that Unix Domain socket would give better performance then TCP loopback connection?

Solution 1

Yes, local interprocess communication by unix domain sockets should be faster than communication by loopback localhost connections because you have less TCP overhead, see here.

Solution 2

This benchmark: https://github.com/rigtorp/ipc-bench provides latency and throughput tests for TCP sockets, Unix Domain Sockets (UDS), and PIPEs.

Here you have the results on a single CPU 3.3GHz Linux machine :

TCP average latency: 6 us

UDS average latency: 2 us

PIPE average latency: 2 us

TCP average throughput: 0.253702 million msg/s

UDS average throughput: 1.733874 million msg/s

PIPE average throughput: 1.682796 million msg/s

66% latency reduction and almost 7X more throughput explain why most performance-critical software has their own IPC custom protocol.

Solution 3

Redis benchmark shows unix domain socket can be significant faster than TCP loopback.

When the server and client benchmark programs run on the same box, both the TCP/IP loopback and unix domain sockets can be used. Depending on the platform, unix domain sockets can achieve around 50% more throughput than the TCP/IP loopback (on Linux for instance). The default behavior of redis-benchmark is to use the TCP/IP loopback.

However, this difference only matters when throughput is high.

Solution 4

Unix domain sockets are often twice as fast as a TCP socket when both peers are on the same host. The Unix domain protocols are not an actual protocol suite, but a way of performing client/server communication on a single host using the same API that is used for clients and servers on different hosts. The Unix domain protocols are an alternative to the interprocess communication (IPC) methods.

View more solutions

Comments

[Janus Troelsen]

the first link is citing the second link, which is from 2005 (old). and it only covers FreeBSD

[easytiger]

This answer is wrong, when tested loopback tcp on modern linux is as fast and sometimes faster than UDS. can provide benchmark if required

[JSON]

This answer is absolutely correct. Loopback interface is still TCP, meaning that you still have the overhead of TCP (congestion control, flow control, stream management (IP packet ordering, retransmission, etc) ). Unix domain sockets do not do any of the above because it was designed from the ground up to be ran locally, meaning no congestion issues, no speed differences between server/client requiring flow control, no dropped packets, etc. Google this if in doubt, not a new thing.

[JSON]

Also note that even local TCP requires every bit to be ACKed. The latency of these acks can be more costly in time than TCP processing.

[CMCDragonkai]

What about local UDP?

[Alexandr Kurilin]

To follow up on what @JSON pointed out, by using unix domain sockets you also bypass the need to tune a lot of TCP-specific configs in sysctl, which removes a bunch of potential bottlenecks that need to be tackled with proper configuration settings and benchmarking.

[GreenReaper]

Sounds to me like their product is an answer to the problem! Maybe that's why they're replying to those questions; because they know an answer.

[ScalaWilliam]

This is a great answer because it has some numbers. Throughput from TCP to UNIX is 350% better, UNIX to PIPE 40% on an i5.

[Dmitry Grigoryev]

@GreenReaper The answer is indeed relevant, but the line our Torusware Speedus product ... come with 2 versions, Speedus Lite and Speedus Extreme Performance (EP) is not, and it makes the whole thing sound like a cheap ad.

[JSON]

Spam. And no, his product isn't relevant in a comparison between TCP and Unix sockets. There's plenty of common sense alternatives to sockets- each outside what the OP is asking

[Yuda Prawira]

Yes, This answer is correct. I was testing benchmark using Unix Domain Socket vs loopback TCP on MariaDB server and I testing it using large query and multithreaded application with a lot of threads. UDS is the winner in the end.

[hanshenrik]

@CMCDragonkai note that local UDP is unsafe. even tho it's local, UDP packets may still be dropped (probably more likely if the system is under heavy load, but still - i've experienced this first-hand, a stable reliable TCP connection became unreliable with packet drops when converted from TCP to UDP, even tho everything was local.)

[Trevor Boyd Smith]

given that the first link is dead (HTTP 404)... this is why stackoverflow best-practice is to at least provide a short/concise relevant quote from the source URL at the time of the answer writing (then when the link goes down the short summary is still available).

[Trevor Boyd Smith]

benchmarks are provided in by scrolling lower: here1 and here2

[falkb]

The usage of that tool is not sufficiently explained. Is there somehow a page explaining how to call client and server?