How fast is state of the art HFT trading systems today?

Question

All the time you hear about high frequency trading (HFT) and how damn fast the algorithms are. But I'm wondering - what is fast these days?

Update

I'm not thinking about the latency caused by the physical distance between an exchange and the server running a trading application, but the latency introduced by the program itself.

To be more specific: What is the time from events arriving on the wire in an application to that application outputs an order/price on the wire? I.e. tick-to-trade time.

Are we talking sub-millisecond? Or sub-microsecond?

How do people achieve these latencies? Coding in assembly? FPGAs? Good-old C++ code?

Update

There's recently been published an interesting article on ACM, providing a lot of details into today's HFT technology, which is an excellent read:

Barbarians at the Gateways - High-frequency Trading and Exchange Technology

Solution 1

You've received very good answers. There's one problem, though - most algotrading is secret. You simply don't know how fast it is. This goes both ways - some may not tell you how fast they work, because they don't want to. Others may, let's say "exaggerate", for many reasons (attracting investors or clients, for one).

Rumors about picoseconds, for example, are rather outrageous. 10 nanoseconds and 0.1 nanoseconds are exactly the same thing, because the time it takes for the order to reach the trading server is so much more than that.

And, most importantly, although not what you've asked, if you go about trying to trade algorithmically, don't try to be faster, try to be smarter. I've seen very good algorithms that can handle whole seconds of latency and make a lot of money.

Solution 2

I'm the CTO of a small company that makes and sells FPGA-based HFT systems. Building our systems on-top of the Solarflare Application Onload Engine (AOE) we have been consistently delivering latency from an "interesting" market event on the wire (10Gb/S UDP market data feed from ICE or CME) to the first byte of the resultant order message hitting the wire in the 750 to 800 nanosecond range (yes, sub-microsecond). We anticipate that our next version systems will be in the 704 to 710 nanosecond range. Some people have claimed slightly less, but that's in a lab environment and not actually sitting at a COLO in Chicago and clearing the orders.

The comments about physics and "speed of light" are valid but not relevant. Everybody that is serious about HFT has their servers at a COLO in the room next to the exchange's server.

To get into this sub-microsecond domain you cannot do very much on the host CPU except feed strategy implementation commands to the FPGA, even with technologies like kernel bypass you have 1.5 microseconds of unavoidable overhead... so in this domain everything is playing with FPGAs.

One of the other answers is very honest in saying that in this highly secretive market very few people talk about the tools they use or their performance. Every one of our clients requires that we not even tell anybody that they use our tools nor disclose anything about how they use them. This not only makes marketing hard, but it really prevents the good flow of technical knowledge between peers.

Because of this need to get into exotic systems for the "wicked fast" part of the market you'll find that the Quants (the folks that come up with the algorithms that we make go fast) are dividing their algos into event-to-response time layers. At the very top of the technology heap are the sub-microsecond systems (like ours). The next layer are the custom C++ systems that make heavy use of kernel bypass and they're in the 3-5 microsecond range. The next layer are the folks that cannot afford to be on a 10Gb/S wire only one router hop from the "exchange", they may be still at COLO's but because of a nasty game we call "port roulette" they're in the dozens to hundreds of microsecond domain. Once you get into milliseconds it's almost not HFT any more.

Cheers

Solution 3

Good article which describes what is the state of HFT (in 2011) and gives some samples of hardware solutions which makes nanoseconds achievable: Wall Streets Need For Trading Speed: The Nanosecond Age

With the race for the lowest “latency” continuing, some market participants are even talking about picoseconds–trillionths of a second.

EDIT: As Nicholas kindly mentioned:

The link mentions a company, Fixnetix, which can "prepare a trade" in 740ns (i.e. the time from an input event occurs to an order being sent).

Solution 4

"sub-40 microseconds" if you want to keep up with Nasdaq. This figure is published here http://www.nasdaqomx.com/technology/

Solution 5

For what its worth, TIBCO's FTL messaging product is sub-500 ns for within a machine (shared memory) and a few micro seconds using RDMA (Remote Direct Memory Access) inside a data center. After that, physics becomes the main part of the equation.

So that is the speed at which data can get from the feed to the app that makes decisions.

At least one system has claimed ~30ns interthread messaging, which is probably a tweaked up benchmark, so anyone talking about lower numbers is using some kind of magic CPU.

Once you are in the app, it is just a question of how fast the program can make decisions.

View more solutions

Comments

[Jorge Y. C. Rodriguez]

hmmm good question! Now i really want to know this!

[sh1]

"On the wire" is kind of a fuzzy boundary. It takes time for a complete data packet to arrive, and some of the processing may already have started before the entire message has been received. Everything is skewed through the different layers of the memory system and the kernel and the application, and people are paying close attention to that skew.

[Nicholas]

Thanks for the insight. Interesting! Can you provide any more details or examples?

[rdalmeida]

The bulk of the latency is usually for network I/O. A good network library should be able to process UDP/TCP with low single digit microsecond latencies. Here are some benchmarks you can check.

[Nicholas]

2010 - that's three years ago! That's eons in this space :-) We could be in the nano-second range by now. Also, it would be great if someone "from the inside" could reveal some figures. Even if it's just old ones from 2012 :)

[Flavio]

Can you provide a more specific link?

[sll]

Right, many companies put hardware in 10meters from stocks not kilometers

[user2485149]

well, if we're playing that game, why not just pay to install an application on the stocks main hubs.. or even add a patch to all the kernels responsible for real time updates.. i think it is a bit of overkill already

[Nicholas]

There's no doubt about overkill being in effect here. But there's real money to earn and I believe some go to extreme to achieve the lowest latency.

[zmbq]

You can't have a lot of hardware 10 meters from the stock exchange servers, there's not a lot of room there...

[Nicholas]

I will accept your answer because I think it has key insights but award the bounty to sll as he delivered the best answer regarding the bounty description. Thanks.

[user3666197]

Brian, you might want to know, that your Linked-In URL fortunately fails to work.

[TomTom]

For 2014 - check goo.gl/3fxqQU for a description of a setup that is smaller.

[bbaassssiiee]

Co-location is the new normal for years. And FPGA's run the algotraders, in hardware.

[rahul.deshmukhpatil]

Mercury-Minerva: what does you mean by "port roulette" here?

[David Arnold]

"port roulette" refers to the assignment of ports to customers by the exchange, in an environment where not all ports have the same hop-distance from the source server(s) and/or the same network hardware. When you request a new port, it's a gamble (the "roulette" part) whether you get a port with good latency or not.

[hmijail]

@Mercury-Minerva, you mention 'latency from an "interesting" market event on the wire' and FPGAs. I was wondering whether there are byte-oriented Ethernet NICs to avoid the 1-frame latency forced by typical frame-at-a-time NICs (which is at least 67 ns in 10GbE). Would you know?

[zmbq]

If the cable connecting your computer to the exchange's computer is 1 meter long, it's going to take the signal at least 3.3 nanoseconds to reach the other side. It's not meaningful when you're talking about 700 nanoseconds, it is meaningful when you talk about 10.

[Vasilios Mavroudis]

10 nanoseconds and 0.1 nanoseconds are exactly the same thing, because the time it takes for the order to reach the trading server is so much more than that. This is a 6 years old comment and I reckon things have changed since. After all, what matters is the relative order of the traders' instructions compared to their competitors. For anyone curious, you can see some stats on reaction times (2018,2019) here.