Why is X11 forwarding so inefficient?

Question

Whenever I remotely launch large GUIs with X11 forwarding, even including the -C switch, the experience is very unresponsive. My question is, what does, at the concept/protocol level cause this?

With my 25mbit connection, I can stream HD video to my computer absolutely without a problem. On the other hand, the unresponsiveness of remotely launched GUIs with X11 forwarding happens even over a 100mbit LAN, where the latency should be near zero.

I understand that as opposed to video streaming, the latency will be at best doubled (as the input needs to be sent to the remote machine and only after that can the appliction respond), but internally, are there other factors which increase the latency even further?

Secondly, the bandwidth. Why does it eat up so much of it? When it comes to picture and video formats, many methods are used to drastically reduce the size.

In the case of .bmp vs .png, for example, a large black square image will take way less in .png representation because information is not stored for every single pixel, but in a range-ish way as far as I understand.

In the case of videos, a whole lot of information can be saved by sending the difference between frames rather than the whole frames.

I know this is very simplified, but is X11 not using these methods? Does it behave in a bitmap-ish or a non-differential principle at some level? And if not, why does it take up so much bandwidth?

Solution 1

The X11 protocol was never meant to handle graphically (in terms of bitmaps/textures) intensive operations. Back in the day when X11 was first designed computer graphics were a lot simpler than they are today.

Basically X11 doesn't send the screen to your computer, but it sends the display-instructions so the X-server on your local computer can re-create the screen on your local system. And this needs to be done on each change/refresh of the display.
So your computer receives a stream of instructions like "draw line in this color from coordinates x,y to (xx,yy), draw rectangle W pixels wide, H pixels high with upper-left corner at (x,y), etc."
The local client isn't really aware what needs to be updated and the remote system has very little information on what the client actually needs, so basically the server must send a lot of redundant information that the client may or may not need.
This is very efficient if the display to be rendered consists of a limited number of simple graphical shapes and only a low refresh frequency (no animations and such) is needed. Which was the case back in the days when X11 was first developed.

But modern GUI's have a lot of eye-candy and much of that needs to be send from the remote system to your client in the form of bitmaps/textures/fonts which take quite a lot of bandwidth. And all sorts of eye-candy includes animated effects requiring frequent updates. And the displays keep getting bigger too, twice as wide/high is 4x the number of pixels.

Of course, over time, enhancements to the X11 protocol were made to optimize this as much as possible, but the basic underlying design is, in essence, simply not well suited to the demands of the kind of GUI's people nowadays expect.

Other protocols (like RDP and VNC) are more designed to let the remote system do all the hard work and let that system decide which updates to send to the client (as compressed bitmaps) as efficiently as possible. Often that turns out to be more efficient for modern GUI's.

Neither method is perfect and can deal with every situation equally well. There is no such thing as a single display-protocol that can do well under every conceivable use-case.
So in most cases you just try all protocols that are supported between your local client and the remote server and use the one that gives the best results. And in some cases there is no choice and you just have to make do with whatever is available.

Most protocols do allow some performance tuning, but many of these settings are server-side only and not available to the average user. (And configuring them properly is a bit of an arcane art. A lot of sys-admins won't be willing to mess with that.)

In most cases the easiest way to improve performance (sometimes quite dramatically) is by switching to a more simple desktop environment with less eye-candy and forego the use of background images.

Solution 2

There are primarily two reasons for X11 connections to be slow, both of which you touched on in your question: bandwidth and latency. To understand why X11 apps are slow over a network, let's discuss both of these.

Bandwidth

X11, by default, doesn't do any compression on the network data that gets passed between the application and the display server. As you mentioned, you can use the -C option on SSH to enable compression, and while this does help, it's not optimized for compressing graphical data. Compared to formats like H.264 which can obtain compression rates of 100 to 1, the -C compression will be lucky to achieve 2 to 1 compression. A better solution is to use a graphics or video optimized codec for the X11 video, but we have to be careful not to make it too lossy as desktops generally need to have sharper images than a movie so the user can still clearly read the text and make out fine details.

Latency

X11 tends to have high latency because most operations require multiple round trips between the application and the display server. When run across a LAN where ping times measure less than a millisecond these multiple round trips aren't noticeable, but across an internet connection they add up quickly.

Solutions

A number of years ago there were a couple projects build to address the bandwidth and latency issues inherent in the X11 protocol. lbx (Low Bandwidth X) and dxpc (Differential X Protocol Compressor). I don't think lbx ever got much traction, but dxpc became the underlying technology used for a product called NX. NX uses both lossy compression to reduce the bandwidth requirements and a differential algorithm and caching to reduce the number of back-and-forth information passing that creates the high latency. I've used NX quite often and find the performance to be almost as good as local applications. If you're feeling up to it you might try NX and see if it works for you. The downside is that it requires installing software on both ends of the connection, whereas X11 is generally already installed.

Comments

[Admin]

Trivia: Xpra offers an interesting approach.

[Admin]

BTW -- using "-C" will slow down your connection if your link is fast enough because compression takes time. "-C" might benefit 100Mb, but likely harm 1Gb, and certainly harm 10Gb. It's also the case that 'ssh' will harm your throughput -- as will any encryption over fast links. If you have a direct-connection between computers or a secure internal-linkage, go direct with your X connection over TCP:6000. You'll get a noticeable speed improvement.

[Admin]

Sounds like you're forwarding through SSH, which has to encrypt / decrypt all of the data. That's going to add overhead / latency. Faster processors might help, but there's a certain amount of latency this will add, no matter WHAT you do. X is very "chatty," so slight increase in latency = significant drop in performance. In times past, I was able to use X, tunneled through SSH over a 28.8 modem; that was using lbxproxy (now deprecated) which cached / compressed a lot of data and reduced the "chattiness" of the connection. Using -C can only add more latency.

[Admin]

Use something like ssh -Y -c blowfish to minimalize overhead while still encrypting. If you have full control of both ends teach ssh to use "none" encryption to get full transfer speed on the connection.

[sleepyweasel]

Tied to the latency topic would be that X11 is going to be TCP, vs UDP for most streaming videos. There are a few other products to help with working remotely. Tony mentioned RDP and VNC. Oracle still sells Sun Global Desktop (SGD) which works well. Citrix had something (XenApp?). Our eval found SGD to be a better option for our needs, but had used two Citrix products prior.

[rath]

+1 Since RDP and VNC are mentioned, I should also mention x2go which is an X11 caching / forwarding solution that really speeds up X11 forwarding. I've used it with success in the past.

[user]

Regarding "server-side only settings" near the end, recall that the X server runs on the computer which is connected to the physical display, and the X client is the software used to perform some task (for example a web browser or word processor). So X server settings would be accessible to the user connecting to the remote system in order to run a graphical application. This does not, however, significantly detract from the value of your answer.

[OrangeDog]

Technically the protocol is RFB, not VNC.

[Jonas Schäfer]

Am I wrong or are you confusing client and server in your second paragraph? The client is the program running remotely, the server is the local machine.

[Joshua]

According to my testing, RDP is not faster than X11. VNC is under many cases because its frame-dropping ability prevents applications from being slowed down by bandwidth limiting framerates.

[Boris the Spider]

NoMachine is also worth a mention, I've found it more responsive and better integrated than VNC.

[Blrfl]

What you covered in the third paragraph was largely mitigated in the 1990s as machines running X servers started to have enough memory that enabling the backing store became a practical thing to do.

[DocSalvager]

In the world of X11, the terms are reversed. The thinking was that the machine with the keyboard/mouse/monitor (i.e. the X-Terminal) provides "presentation" services to the machine running the main program. In this case, the remote host server is the "X-client".

[comte]

x2go worked very fine for me, even with "server" an old laptop. up&running in few minutes... big increase in performance from X11 fwd (unusable with my config)

[rakslice]

Latency wise, on *ix machines, X11 sessions to local displays usually use Unix domain sockets instead of TCP; Unix domain sockets are many times faster than TCP at round trips even to localhost stackoverflow.com/questions/14973942/…. For X11 apps with really pathologically large numbers of round trips, that could be the difference between okay and noticeably slow performance.

[i486]

I don't have deep knowledge of X11 but I guess whether libraries like GTK/Qt may be additional reason for slow remote work?

[Gerry]

Ahhh I never knew this background to NX. I've always been amazed by how fast it is. I just wish it was more mainstream as it doesn't look like it gets much attention any more.

[9a3eedi]

I have found that X11 is extremely sensitive to latency. Even switching from a fast wifi connection to a fast ethernet connection, despite having similar bandwidth, can make a very big difference, simply because wifi has a lot more jitter in the latency