C++ view types: pass by const& or by value?

Question

This came up in a code review discussion recently, but without a satisfactory conclusion. The types in question are analogues to the C++ string_view TS. They are simple non-owning wrappers around a pointer and a length, decorated with some custom functions:

#include <cstddef>

class foo_view {
public:
    foo_view(const char* data, std::size_t len)
        : _data(data)
        , _len(len) {
    }

    // member functions related to viewing the 'foo' pointed to by '_data'.

private:
    const char* _data;
    std::size_t _len;
};

The question arose as to whether there is an argument either way to prefer to pass such view types (including the upcoming string_view and array_view types) by value or by const reference.

Arguments in favor of pass by value amounted to 'less typing', 'can mutate the local copy if the view has meaningful mutations', and 'probably no less efficient'.

Arguments in favor of pass-by-const-reference amounted to 'more idiomatic to pass objects by const&', and 'probably no less efficient'.

Are there any additional considerations that might swing the argument conclusively one way or the other in terms of whether it is better to pass idiomatic view types by value or by const reference.

For this question it is safe to assume C++11 or C++14 semantics, and sufficiently modern toolchains and target architectures, etc.

Solution 1

When in doubt, pass by value.

Now, you should only rarely be in doubt.

Often values are expensive to pass and give little benefit. Sometimes you actually want a reference to a possibly mutating value stored elsewhere. Often, in generic code, you don't know if copying is an expensive operation, so you err on the side of not.

The reason why you should pass by value when in doubt is because values are easier to reason about. A reference (even a const one) to external data could mutate in the middle of an algorithm when you call a function callback or what have you, rendering what seems to be a simple function into a complex mess.

In this case, you already have an implicit reference bind (to the contents of the container you are viewing). Adding another implicit reference bind (to the view object that looks into the container) is no less bad because there are already complications.

Finally, compilers can reason about values better than they can about references to values. If you leave the locally analyzed scope (via a function pointer callback), the compiler has to presume the value stored in the const reference may have completely changed (if it cannot prove the contrary). A value in automatic storage with nobody taking a pointer to it can be assumed to not modify in a similar way -- there is no defined way to access it and change it from an external scope, so such modifications can be presumed to not-happen.

Embrace the simplicity when you have an opportunity to pass a value as a value. It only happens rarely.

Solution 2

EDIT: Code is available here: https://github.com/acmorrow/stringview_param

I've created some example code which appears to demonstrate that pass-by-value for string_view like objects results in better code for both callers and function definitions on at least one platform.

First, we define a fake string_view class (I didn't have the real thing handy) in string_view.h:

#pragma once

#include <string>

class string_view {
public:
    string_view()
        : _data(nullptr)
        , _len(0) {
    }

    string_view(const char* data)
        : _data(data)
        , _len(strlen(data)) {
    }

    string_view(const std::string& data)
        : _data(data.data())
        , _len(data.length()) {
    }

    const char* data() const {
        return _data;
    }

    std::size_t len() const {
        return _len;
    }

private:
    const char* _data;
    size_t _len;
};

Now, lets define some functions that consume a string_view, either by value or by reference. Here are the signatures in example.hpp:

#pragma once

class string_view;

void __attribute__((visibility("default"))) use_as_value(string_view view);
void __attribute__((visibility("default"))) use_as_const_ref(const string_view& view);

The bodies of these functions are defined as follows, in example.cpp:

#include "example.hpp"

#include <cstdio>

#include "do_something_else.hpp"
#include "string_view.hpp"

void use_as_value(string_view view) {
    printf("%ld %ld %zu\n", strchr(view.data(), 'a') - view.data(), view.len(), strlen(view.data()));
    do_something_else();
    printf("%ld %ld %zu\n", strchr(view.data(), 'a') - view.data(), view.len(), strlen(view.data()));
}

void use_as_const_ref(const string_view& view) {
    printf("%ld %ld %zu\n", strchr(view.data(), 'a') - view.data(), view.len(), strlen(view.data()));
    do_something_else();
    printf("%ld %ld %zu\n", strchr(view.data(), 'a') - view.data(), view.len(), strlen(view.data()));
}

The do_something_else function is here is a stand-in for arbitrary calls to functions that the compiler does not have insight about (e.g. functions from other dynamic objects, etc.). The declaration is in do_something_else.hpp:

#pragma once

void __attribute__((visibility("default"))) do_something_else();

And the trivial definition is in do_something_else.cpp:

#include "do_something_else.hpp"

#include <cstdio>

void do_something_else() {
    std::printf("Doing something\n");
}

We now compile do_something_else.cpp and example.cpp into individual dynamic libraries. Compiler here is XCode 6 clang on OS X Yosemite 10.10.1:

clang++ -mmacosx-version-min=10.10 --stdlib=libc++ -O3 -flto -march=native -fvisibility-inlines-hidden -fvisibility=hidden --std=c++11 ./do_something_else.cpp -fPIC -shared -o libdo_something_else.dylib clang++ -mmacosx-version-min=10.10 --stdlib=libc++ -O3 -flto -march=native -fvisibility-inlines-hidden -fvisibility=hidden --std=c++11 ./example.cpp -fPIC -shared -o libexample.dylib -L. -ldo_something_else

Now, we disassemble libexample.dylib:

> otool -tVq ./libexample.dylib
./libexample.dylib:
(__TEXT,__text) section
__Z12use_as_value11string_view:
0000000000000d80    pushq   %rbp
0000000000000d81    movq    %rsp, %rbp
0000000000000d84    pushq   %r15
0000000000000d86    pushq   %r14
0000000000000d88    pushq   %r12
0000000000000d8a    pushq   %rbx
0000000000000d8b    movq    %rsi, %r14
0000000000000d8e    movq    %rdi, %rbx
0000000000000d91    movl    $0x61, %esi
0000000000000d96    callq   0xf42                   ## symbol stub for: _strchr
0000000000000d9b    movq    %rax, %r15
0000000000000d9e    subq    %rbx, %r15
0000000000000da1    movq    %rbx, %rdi
0000000000000da4    callq   0xf48                   ## symbol stub for: _strlen
0000000000000da9    movq    %rax, %rcx
0000000000000dac    leaq    0x1d5(%rip), %r12       ## literal pool for: "%ld %ld %zu\n"
0000000000000db3    xorl    %eax, %eax
0000000000000db5    movq    %r12, %rdi
0000000000000db8    movq    %r15, %rsi
0000000000000dbb    movq    %r14, %rdx
0000000000000dbe    callq   0xf3c                   ## symbol stub for: _printf
0000000000000dc3    callq   0xf36                   ## symbol stub for: __Z17do_something_elsev
0000000000000dc8    movl    $0x61, %esi
0000000000000dcd    movq    %rbx, %rdi
0000000000000dd0    callq   0xf42                   ## symbol stub for: _strchr
0000000000000dd5    movq    %rax, %r15
0000000000000dd8    subq    %rbx, %r15
0000000000000ddb    movq    %rbx, %rdi
0000000000000dde    callq   0xf48                   ## symbol stub for: _strlen
0000000000000de3    movq    %rax, %rcx
0000000000000de6    xorl    %eax, %eax
0000000000000de8    movq    %r12, %rdi
0000000000000deb    movq    %r15, %rsi
0000000000000dee    movq    %r14, %rdx
0000000000000df1    popq    %rbx
0000000000000df2    popq    %r12
0000000000000df4    popq    %r14
0000000000000df6    popq    %r15
0000000000000df8    popq    %rbp
0000000000000df9    jmp 0xf3c                   ## symbol stub for: _printf
0000000000000dfe    nop
__Z16use_as_const_refRK11string_view:
0000000000000e00    pushq   %rbp
0000000000000e01    movq    %rsp, %rbp
0000000000000e04    pushq   %r15
0000000000000e06    pushq   %r14
0000000000000e08    pushq   %r13
0000000000000e0a    pushq   %r12
0000000000000e0c    pushq   %rbx
0000000000000e0d    pushq   %rax
0000000000000e0e    movq    %rdi, %r14
0000000000000e11    movq    (%r14), %rbx
0000000000000e14    movl    $0x61, %esi
0000000000000e19    movq    %rbx, %rdi
0000000000000e1c    callq   0xf42                   ## symbol stub for: _strchr
0000000000000e21    movq    %rax, %r15
0000000000000e24    subq    %rbx, %r15
0000000000000e27    movq    0x8(%r14), %r12
0000000000000e2b    movq    %rbx, %rdi
0000000000000e2e    callq   0xf48                   ## symbol stub for: _strlen
0000000000000e33    movq    %rax, %rcx
0000000000000e36    leaq    0x14b(%rip), %r13       ## literal pool for: "%ld %ld %zu\n"
0000000000000e3d    xorl    %eax, %eax
0000000000000e3f    movq    %r13, %rdi
0000000000000e42    movq    %r15, %rsi
0000000000000e45    movq    %r12, %rdx
0000000000000e48    callq   0xf3c                   ## symbol stub for: _printf
0000000000000e4d    callq   0xf36                   ## symbol stub for: __Z17do_something_elsev
0000000000000e52    movq    (%r14), %rbx
0000000000000e55    movl    $0x61, %esi
0000000000000e5a    movq    %rbx, %rdi
0000000000000e5d    callq   0xf42                   ## symbol stub for: _strchr
0000000000000e62    movq    %rax, %r15
0000000000000e65    subq    %rbx, %r15
0000000000000e68    movq    0x8(%r14), %r14
0000000000000e6c    movq    %rbx, %rdi
0000000000000e6f    callq   0xf48                   ## symbol stub for: _strlen
0000000000000e74    movq    %rax, %rcx
0000000000000e77    xorl    %eax, %eax
0000000000000e79    movq    %r13, %rdi
0000000000000e7c    movq    %r15, %rsi
0000000000000e7f    movq    %r14, %rdx
0000000000000e82    addq    $0x8, %rsp
0000000000000e86    popq    %rbx
0000000000000e87    popq    %r12
0000000000000e89    popq    %r13
0000000000000e8b    popq    %r14
0000000000000e8d    popq    %r15
0000000000000e8f    popq    %rbp
0000000000000e90    jmp 0xf3c                   ## symbol stub for: _printf
0000000000000e95    nopw    %cs:(%rax,%rax)

Interestingly, the by-value version is several instructions shorter. But that is only the function bodies. What about callers?

We will define some functions that invoke these two overloads, forwarding a const std::string&, in example_users.hpp:

#pragma once

#include <string>

void __attribute__((visibility("default"))) forward_to_use_as_value(const std::string& str);
void __attribute__((visibility("default"))) forward_to_use_as_const_ref(const std::string& str);

And define them in example_users.cpp:

#include "example_users.hpp"

#include "example.hpp"
#include "string_view.hpp"

void forward_to_use_as_value(const std::string& str) {
    use_as_value(str);
}

void forward_to_use_as_const_ref(const std::string& str) {
    use_as_const_ref(str);
}

Again, we compile example_users.cpp to a shared library:

clang++ -mmacosx-version-min=10.10 --stdlib=libc++ -O3 -flto -march=native -fvisibility-inlines-hidden -fvisibility=hidden --std=c++11 ./example_users.cpp -fPIC -shared -o libexample_users.dylib -L. -lexample

And, again, we look at the generated code:

> otool -tVq ./libexample_users.dylib
./libexample_users.dylib:
(__TEXT,__text) section
__Z23forward_to_use_as_valueRKNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEE:
0000000000000e70    pushq   %rbp
0000000000000e71    movq    %rsp, %rbp
0000000000000e74    movzbl  (%rdi), %esi
0000000000000e77    testb   $0x1, %sil
0000000000000e7b    je  0xe8b
0000000000000e7d    movq    0x8(%rdi), %rsi
0000000000000e81    movq    0x10(%rdi), %rdi
0000000000000e85    popq    %rbp
0000000000000e86    jmp 0xf60                   ## symbol stub for: __Z12use_as_value11string_view
0000000000000e8b    incq    %rdi
0000000000000e8e    shrq    %rsi
0000000000000e91    popq    %rbp
0000000000000e92    jmp 0xf60                   ## symbol stub for: __Z12use_as_value11string_view
0000000000000e97    nopw    (%rax,%rax)
__Z27forward_to_use_as_const_refRKNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEE:
0000000000000ea0    pushq   %rbp
0000000000000ea1    movq    %rsp, %rbp
0000000000000ea4    subq    $0x10, %rsp
0000000000000ea8    movzbl  (%rdi), %eax
0000000000000eab    testb   $0x1, %al
0000000000000ead    je  0xebd
0000000000000eaf    movq    0x10(%rdi), %rax
0000000000000eb3    movq    %rax, -0x10(%rbp)
0000000000000eb7    movq    0x8(%rdi), %rax
0000000000000ebb    jmp 0xec7
0000000000000ebd    incq    %rdi
0000000000000ec0    movq    %rdi, -0x10(%rbp)
0000000000000ec4    shrq    %rax
0000000000000ec7    movq    %rax, -0x8(%rbp)
0000000000000ecb    leaq    -0x10(%rbp), %rdi
0000000000000ecf    callq   0xf66                   ## symbol stub for: __Z16use_as_const_refRK11string_view
0000000000000ed4    addq    $0x10, %rsp
0000000000000ed8    popq    %rbp
0000000000000ed9    retq
0000000000000eda    nopw    (%rax,%rax)

And, again, the by-value version is several instructions shorter.

It appears to me that, at least by the coarse metric of instruction count, that the by-value version produces better code for both callers and for generated function bodies.

I'm of course open to suggestions to how to improve this test. Obviously a next step would be to refactor this into something where I could benchmark it meaningfully. I will try to do that soon.

I will post the example code to github with some sort of build script so others can test on their systems.

But based on the discussion above, and the results of inspecting the generated code, my conclusion is that pass-by-value is the way to go for view types.

Solution 3

Putting aside philosophical questions about the signaling value of const&-ness versus value-ness as function parameters, we can take a look at some ABI implications on various architectures.

http://www.macieira.org/blog/2012/02/the-value-of-passing-by-value/ lays out some decision making and testing done by some QT folks on x86-64, ARMv7 hard-float, MIPS hard-float (o32) and IA-64. Mostly, it checks whether functions can pass various structs through registers. Not surprisingly, it appears that each platform can manage 2 pointers by register. And given that sizeof(size_t) is generally sizeof(void*), there's little reason to believe that we'll be spilling to memory here.

We can find more wood for the fire, considering suggestions like: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3538.html. Note that const ref has some downsides, namely the risk of aliasing, which can prevent important optimizations and require extra thought for the programmer. In the absence of C++ support for C99's restrict, passing by value can improve performance and lower cognitive load.

I suppose then that I'm synthesizing two arguments in favor of pass by value:

1. 32 bit platforms often lacked the capacity to pass two word structs by register. This no longer appears to be an issue.
2. const references are quantitatively and qualitatively worse than values in that they can alias.

All of which would lead me to favor pass by value for <16 byte structures of integral types. Obviously your mileage may vary and testing should always be done where performance is an issue, but values do seem a little nicer for very small types.

Solution 4

In addition to what have already been said here in favour of passing by value, modern C++ optimizers struggle with reference arguments.

When the body of the callee is not available in the translation unit (the function resides in a shared library or in another translation unit and link-time optimization is not available) the following things happen:

1. The optimizer assumes that the arguments passed by reference or reference to const can be changed (const does not matter because of const_cast) or referred to by a global pointer, or changed by another thread. Basically, the arguments passed by reference become poisoned values in the call site, which the optimizer cannot apply many optimizations to any more.
2. In the callee if there are several reference/pointer arguments of the same base type, the optimizer assumes that they alias with something else and that again precludes many optimizations.
3. Moreover, all char type arrays can alias values of any other type, so that modifying any std::string object implies modifying any and all other objects, resulting in following machine code having to reload all objects from memory. restrict keyword was added to C to solve exactly this inefficiency. Different addresses may still alias because one can map one page frame multiple times into one virtual address space (and that's a popular trick for 0-copy receive ring buffers), that's why the compiler cannot assume no aliasing for different addresses unless restrict keyword is used.

From the optimizer standpoint of view passing and returning by value is the best because that obviates the need for alias analysis: the caller and the callee own their copies of values exclusively so that these values cannot be modified from anywhere else.

For a detailed treatment of the subject I cannot recommend enough Chandler Carruth: Optimizing the Emergent Structures of C++. The punchline of the talk is "people need to change their heads about passing by value... the register model of passing arguments is obsolete."

Solution 5

Here are my rules of thumb for passing variables to functions:

1. If the variable can fit inside the processor's register and will not be modified, pass by value.
2. If the variable will be modified, pass by reference.
3. If the variable is larger than the processor's register and will not be modified, pass by constant reference.
4. If you need to use pointers, pass by smart pointer.

Hope that helps.

View more solutions

Comments

[Admin]

You might correct #2 (modification inside the function, modified (out) argument)

[acm]

I understand the difference, and was not asking for clarification on the semantics of function parameters.

[James Kanze]

What's the logic behind #4. As far as I can tell, you almost never want to pass a smart pointer as argument.

[acm]

I didn't say it held up the code review. It was a side discussion. My question is not about correctness, but idiom.

[6502]

If you know they're different concepts then it makes no sense to ask if you should use one or the other. You need both, depending on the context. Unfortunately C++ pushes the idea that a const T& is a cool way to say T and mistakes of this kind are even in the standard library itself.

[Félix Cantournet]

@JamesKanze You would use 4 if the ownership is shared/mutable and the memmory is heap allocated.

[Paul J. Lucas]

I personally don't like #2 since you can't tell by looking at the code whether an argument is modified. If an argument is modified, I always pass by pointer. (Hence, I don't like non-const references for arguments.)

[acm]

F(T t) and F(const T& t) have very similar, though of course not identical, semantics. As someone writing an F for which either declaration is appropriate for the intended implementation, I must make a choice between these declarations. I am asking whether, when both are appropriate, there is a reason to prefer one choice over the other, for a narrow class of types (pointer + size).

[acm]

I like the argument about compilers reasoning about values better. The point about the compiler needing to assume that any callback may have mutated the referred to object seems rather convincing to me.

[acm]

Yes, but you can always declare it to take a const value.

[James Kanze]

@FélixCantournet Maybe, maybe not. If the called function should share in the ownership, perhaps. But shared ownership is rather rare to begin with.

[Andre Kostur]

@acm True. Then back to the consideration of the cost of construction. If it's expensive to construct, then a const& is probably a better idea. (Putting aside multithreading issues. That adds a whole other dimension of considerations.)

[6502]

@acm: The semantic is indeed very different; in the F(const T&) you're communicating the address of an object and F can access current and future states, with F(T) you're communicating just the current state. If you check the rect example in the linked answer you should notice how using -=(const P2d&) is conceptually wrong (and it bites back). The only excuse for using const references for values is efficiency but in your case doesn't hold, with values the compiler will do a better job.

[acm]

We seem to not be communicating well. I am aware that passing a reference is communicating the address. My point is that many function bodies would a) continue to compile and b) mean the same thing, if the type of the parameter was changed from T to const T&, and vice versa. Therefore, there is a large body of functions for which the choice is semantically arbitrary.

[acm]

Right. In general I agree with you. Especially for an arbitrary type T in a template, where you can't know the cost of passing T by value, T const& is very likely always preferable. However, I'm interested in the particular case where we both know the exact type, and we know that the representation of that type is precisely one pointer and one size_t.

[Thomas Matthews]

Large objects should not be passed by value. Pointers and references were created to prevent unnecessary copying of large structures / objects.

[acm]

@ThomasMatthews The question is explicitly about small objects.

[6502]

@acm: Like I wrote already specifying a const T& when the callee is not interested in the identity (i.e. future state changes) is a design error. The only justification for that is when the object is heavy and making a copy is a serious performance problem. For small objects making copies is often actually better from a performance point of view because there is one indirection less and the optimizer paranoid side doesn't need to consider aliasing problems (e.g. if you have F(const X& a, X& b) the optimizer will consider the possibility that both references are to the same object).

[acm]

@LightnessRacesinOrbit Yes, for me as well. Based on the above comment, and the results of my codegen experiments below, the original code review discussion seems to have been settled strongly in favor of view types always passed by value.

[newacct]

"Since it doesn't make the slightest difference which one you use in this case" What makes you say that?

[miritovil]

Dumb pointers are excellent if there's no ownership semantics involved. If the function will retain the pointer, such that it can be accessed at some time after returning, passing smart pointers should be required. Otherwise, passing a dumb pointer is good.

[seand]

I agree passing small objects by value is best. What's small? Perhaps under 128 bits? Using &const for small objects will cause an indirection that's likely to slow things down.

[oliora]

Answer by @acm is more correct answer. It has some actual proof for passing by value.

[KeyC0de]

I would add "When in doubt, pass and return by value."

[underscore_d]

Given your point #1, would you say that on 64-bit platforms, less than or equal to 16 byte structures should probably be passed by value rather than reference? That tends to be my rule-of-thumb: 2 words, however long those are on the platform in question. This is based on an assumption that 2 words either can fit in an extended register, or is cheap enough to copy that indirection is worse anyway.

[underscore_d]

"take it by value. But this way you are explicitly communicating to future developers that you intend to modify the instance." ...What? No, you're not. You're only communicating to them that you'll take a copy of what they pass. What you do with it is none of their business because it's a copy of theirs. If you don't want to modify it, then declare the argument const in the implementation. That should be the habit unless you actually need to modify the copy. But, either way, you have a different instance, which a caller has no business caring about.

[Jouni]

I don't understand the phrase "value stored in the const reference may have completely changed". Why if it is CONST reference?

[Yakk - Adam Nevraumont]

@Jouny const applies to the access path, not the actual constness of the value. You aren't permitted (without a const_cast) to edit a const value, but anyone else who holds a reference or pointer to it can, and your view of the value has to reflect that change. This means that (a) the compiler has to reload the data every time it leaves local scope of the function, and (b) the programmer has to deal with the possibility the value is changing under their feet non-locally.

[Slava]

"F(const T&) you're communicating the address of an object and F can access current and future states, with F(T) you're communicating just the current state." Unfortunately C++ does not distinguish this semantic difference from mere optimization to avoid making a copy of large object. This is the source of many bugs and of my biggest issues with the lang.