Virtual functions and performance - C++

Question

In my class design, I use abstract classes and virtual functions extensively. I had a feeling that virtual functions affects the performance. Is this true? But I think this performance difference is not noticeable and looks like I am doing premature optimization. Right?

Solution 1

A good rule of thumb is:

It's not a performance problem until you can prove it.

The use of virtual functions will have a very slight effect on performance, but it's unlikely to affect the overall performance of your application. Better places to look for performance improvements are in algorithms and I/O.

An excellent article that talks about virtual functions (and more) is Member Function Pointers and the Fastest Possible C++ Delegates.

Solution 2

Your question made me curious, so I went ahead and ran some timings on the 3GHz in-order PowerPC CPU we work with. The test I ran was to make a simple 4d vector class with get/set functions

class TestVec 
{
    float x,y,z,w; 
public:
    float GetX() { return x; }
    float SetX(float to) { return x=to; }  // and so on for the other three 
}

Then I set up three arrays each containing 1024 of these vectors (small enough to fit in L1) and ran a loop that added them to one another (A.x = B.x + C.x) 1000 times. I ran this with the functions defined as inline, virtual, and regular function calls. Here are the results:

• inline: 8ms (0.65ns per call)
• direct: 68ms (5.53ns per call)
• virtual: 160ms (13ns per call)

So, in this case (where everything fits in cache) the virtual function calls were about 20x slower than the inline calls. But what does this really mean? Each trip through the loop caused exactly 3 * 4 * 1024 = 12,288 function calls (1024 vectors times four components times three calls per add), so these times represent 1000 * 12,288 = 12,288,000 function calls. The virtual loop took 92ms longer than the direct loop, so the additional overhead per call was 7 nanoseconds per function.

From this I conclude: yes, virtual functions are much slower than direct functions, and no, unless you're planning on calling them ten million times per second, it doesn't matter.

See also: comparison of the generated assembly.

Solution 3

When Objective-C (where all methods are virtual) is the primary language for the iPhone and freakin' Java is the main language for Android, I think it's pretty safe to use C++ virtual functions on our 3 GHz dual-core towers.

Solution 4

In very performance critical applications (like video games) a virtual function call can be too slow. With modern hardware, the biggest performance concern is the cache miss. If data isn't in the cache, it may be hundreds of cycles before it's available.

A normal function call can generate an instruction cache miss when the CPU fetches the first instruction of the new function and it's not in the cache.

A virtual function call first needs to load the vtable pointer from the object. This can result in a data cache miss. Then it loads the function pointer from the vtable which can result in another data cache miss. Then it calls the function which can result in an instruction cache miss like a non-virtual function.

In many cases, two extra cache misses are not a concern, but in a tight loop on performance critical code it can dramatically reduce performance.

Solution 5

From page 44 of Agner Fog's "Optimizing Software in C++" manual:

The time it takes to call a virtual member function is a few clock cycles more than it takes to call a non-virtual member function, provided that the function call statement always calls the same version of the virtual function. If the version changes then you will get a misprediction penalty of 10 - 30 clock cycles. The rules for prediction and misprediction of virtual function calls is the same as for switch statements...

View more solutions

Comments

[Suma]

As per my answer, I suggest closing this as duplicate of stackoverflow.com/questions/113830

[Bo Persson]

possible duplicate of Performance penalty for working with interfaces in C++?

[Vincent]

If you are doing high performance computing and number crunching, do not use any virtuality in the core of calculation : it definitely kills all performances and prevent optimizations at compile-time. For initialization or finalization of the program it is not important. When working with interfaces, you can use virtuality as you wish.

[puio]

quick-bench.com/q/hU7VjdB0IP7rxjYuH46xbocVBxY Try this benchmark. 10% difference in a tight loop. 20% in a single call quick-bench.com/q/Y4FvX3adXOjVp3Bh2SmbG-jVtco

[Suma]

The article linked does not consider very important part of virtual call, and that is possible branch misprediction.

[Greg Rogers]

calling anything in a tight loop is likely to keep all that code and data hot in cache...

[Dominik Grabiec]

Yes, but if that right loop is iterating through a list of objects then each object could potentially be calling a virtual function at a different address through the same function call.

[Sebastian Mach]

But if if they are called multiple times, they can often be cheaper than when only called one time. See my irrelevant blog: phresnel.org/blog , the posts titled "Virtual functions considered not harmful", but of course it depends on the complexity of your codepaths

[Crashworks]

My test measures a small set of virtual functions called repeatedly. Your blog post assumes that the time-cost of code can be measured by counting operations, but that isn't always true; the major cost of a vfunc on modern processors is the pipeline bubble caused by a branch mispredict.

[Crashworks]

I'm not sure the iPhone is a good example of performant code: youtube.com/watch?v=Pdk2cJpSXLg

[Chuck]

@Crashworks: The iPhone is not an example of code at all. It's an example of hardware — specifically slow hardware, which is the point I was making here. If these reputedly "slow" languages are good enough for underpowered hardware, virtual functions are not going to be a huge problem.

[Qwertie]

Right, but any code (or vtable) that is called repeatedly from a tight loop will (of course) rarely suffer cache misses. Besides, the vtable pointer is typically in the same cache line as other data in the object that the called method will access, so often we're talking about only one extra cache miss.

[Qwertie]

Well, even today avoiding function calls is important for high-perf apps. The difference is, today's compilers reliably inline small functions so we don't suffer speed penalties for writing small functions. As for virtual functions, smart CPUs can do smart branch prediction on them. The fact that old computers were slower is, I think, not really the issue--yes, they were much slower, but back then we knew that, so we gave them much smaller workloads. In 1992 if we played an MP3, we knew we might have to dedicate over half of the CPU to that task.

[Bjarke H. Roune]

I think it is quite likely that your compiler can tell that the virtual function call in your code can only call Virtual::call. In that case it can just inline it. There is also nothing preventing the compiler from inlining Normal::call even though you didn't ask it to. So I think that it is quite possible that you get the same times for the 3 operations because the compiler is generating identical code for them.

[lurscher]

this would be a great benchmark for gcc LTO (Link Time Optimization); try to compile this again with lto enabled: gcc.gnu.org/wiki/LinkTimeOptimization and see what happens with the 20x factor

[HaltingState]

The iPhone runs on an ARM processor. The ARM processors used for iOS are designed for low MHz and low power usage. There is no silicon for branch prediction on the CPU and therefore no performance overhead from branch prediction misses from virtual function calls. Also the MHz for iOS hardware is low enough that a cache miss does not stall processor for 300 clock cycles while it retrieves data from RAM. Cache misses are less important at lower MHz. In short, there is no overhead from using virtual functions on iOS devices, but this is a hardware issue and does not apply to desktops CPUs.

[Bo Persson]

A virtual function call is expensive compared to ++ia. So what?

[Arto Bendiken]

Thanks for this reference. Agner Fog's optimization manuals are the gold standard for optimally utilizing hardware.

[Ghita]

@Qwertie I don't think that's necessary true. The body of the loop (if larger than L1 cache) could "retire" vtable pointer, function pointer and subsequent iteration would have to wait for L2 cache (or more) access on every iteration

[thomthom]

What about pure virtual functions? Do they affect performance in any way? Just wondering as it seem that they are there simply to enforce implementation.

[Greg Hewgill]

@thomthom: Correct, there is no performance difference between pure virtual and ordinary virtual functions.

[chew socks]

Do you mind releasing the code for your benchmark? I'm sure some of us would be interested in comparing it on different hardware.

[Crashworks]

@chewsocks It's at assemblyrequired.crashworks.org/2009/01/19/…

[v.oddou]

mp3 dates from 1995. in 92 we barely had 386, no way they could play an mp3, and 50% of cpu time assumes a good multi task OS, an idle process, and a preemptive scheduler. None of this existed on consumer market at the time. it was 100% from the moment power was ON, end of story.

[thomthom]

If a class has one virtual and one inline function, will the performance of the non-virtual method also be affected? Simply by the nature of the class being virtual?

[Crashworks]

@thomthom No, virtual/non-virtual is a per-function attribute. A function only needs to be defined via vtable if it's marked as virtual or if it's overriding a base class that has it as virtual. You'll often see classes that have a group of virtual functions for public interface, and then a lot of inline accessors and so on. (Technically, this is implementation-specific and a compiler could use virtual ponters even for functions marked 'inline', but a person who wrote such a compiler would be insane.)

[zumalifeguard]

never ever, no matter how small the target computer?

[Alex Suo]

As a long time Java programmer newly into C++, I wanna add that Java's JIT compiler and run-time optimizer has the ability to compile, predict and even inline some functions at run-time after a predefined number of loops. However I am not sure if C++ has such feature at compile and link time because it lacks runtime call pattern. Thus in C++ we might need to be slightly more careful.

[underscore_d]

@AlexSuo I'm not sure of your point? Being compiled, C++ of course cannot optimise based on what might happen at runtime, so prediction etc would have to be done by the CPU itself... but good C++ compilers (if instructed) go to great lengths to optimise functions and loops long before runtime.

[underscore_d]

Based on my recollection and a quick search - stackoverflow.com/questions/17061967/c-switch-and-jump-table‌​s - I doubt this is always true for switch. With totally arbitrary case values, sure. But if all cases are consecutive, a compiler might be able to optimise this into a jump-table (ah, that reminds me of the good old Z80 days), which should be (for want of a better term) constant-time. Not that I recommend trying to replace vfuncs with switch, which is ludicrous. ;)

[underscore_d]

I might have agreed had you phrased that as The performance penalty of using virtual functions can sometimes be so insignificant that it is completely outweighed by the advantages you get at the design level. The key difference is saying sometimes, not never.

[tjysdsg]

It's called Parametric polymorphism

[puio]

quick-bench.com/q/hU7VjdB0IP7rxjYuH46xbocVBxY Here's a benchmark which shows just 10% difference.

[HCSF]

@underscore_d I think you are right that the vtable could be optimized to a jump table but what Agner's statement about rules for prediction and misprediction of virtual function calls is the same as for switch statements is also true in a sense that let's say vtable is implemented as a switch-case, then there are two possibilities: 1) it gets optimized to a jump table (as you said) if the cases are consecutive, 2) it can't be optimized to a jump table because the cases aren't consecutive, and so will get a misprediction penalty of 10 - 30 clock cycles as Anger states.

[Tara]

Considering the recent push for more cache-friendly programming, as memory speeds cannot keep up with CPU speeds, this answer is bad advice.