Using C++11 range-based for loop correctly in Qt

Question

According to this talk there is a certain pitfall when using C++11 range base for on Qt containers. Consider:

QList<MyStruct> list;
for(const MyStruct &item : list)
{
    //...
}

The pitfall, according to the talk, comes from the implicit sharing. Under the hood the ranged-based for gets the iterator from the container. But because the container is not const the iterator will be non-const and that is apparently enough for the container to detach.

When you control the lifetime of a container this is easy to fix, one just passes the const reference to the container to force it to use const_iterator and not to detach.

QList<MyStruct> list;
const Qlist<MyStruct> &constList = list;
for(const MyStruct &item : constList)
{
    //...
}

However what about for example containers as return values.

QList<MyStruct> foo() { //... }
void main()
{
    for(const MyStruct &item : foo())
    {
    }
}

What does happen here? Is the container still copied? Intuitively I would say it is so to avoid that this might need to be done?

QList<MyStruct> foo() { //... }
main()
{ 
    for(const MyStruct &item : const_cast<const QList<MyStruct>>(foo()))
    {
    }
}

I am not sure. I know it is a bit more verbose but I need this because I use ranged based for loops heavily on huge containers a lot so the talk kind of struck the right string with me.

So far I use a helper function to convert the container to the const reference but if there is a shorter/easier way to achieve the same I would like to hear it.

Solution 1

template<class T>
std::remove_reference_t<T> const& as_const(T&&t){return t;}

might help. An implicitly shared object returned an rvalue can implicitly detect write-shraring (and detatch) due to non-const iteration.

This gives you:

for(auto&&item : as_const(foo()))
{
}

which lets you iterate in a const way (and pretty clearly).

If you need reference lifetime extension to work, have 2 overloads:

template<class T>
T const as_const(T&&t){return std::forward<T>(t);}
template<class T>
T const& as_const(T&t){return t;}

But iterating over const rvalues and caring about it is often a design error: they are throw away copies, why does it matter if you edit them? And if you behave very differently based off const qualification, that will bite you elsewhere.

Solution 2

Qt has an implementation to resolve this, qAsConst (see https://doc.qt.io/qt-5/qtglobal.html#qAsConst). The documentation says that it is Qt's version of C++17's std::as_const().

Comments

[Dmitry Sazonov]

Stop worrying about that. All Qt containers implements COW pattern. And in latest versions Qt team implements support of C++11, including move ctors.

[Dmitry Sazonov]

Btw, try const MyStruct& const item : foo() to iterate in const style.

[Resurrection]

@SaZ I will try your suggestion. But regarding COW the Qt developer in the linked talk explicitly said that creating non-const iterator from a container means it detaches. It makes sense because otherwise they could not detect if you actually did use that iterator to change it, simply the fact you can is enough.

[AngryDuck]

i have literally never had a problem with just doing for(const auto& bla : blas) i dont see there could be a problem with this even

[avb]

Shouldn't it be const QList<MyStruct> &constList = list; instead of Qlist<MyStruct> &constList = list; to get const iterators and prevent detach? If no, why not?

[Resurrection]

@avb Yes of course, thanks for noticing that!

[avb]

Okay now it makes sense to me :) Now wouldn't QList<MyStruct> &constList = foo(); and then for(const MyStruct &item : constList) solve your initial problem?

[Resurrection]

@avb No because that would also detach in the loop.

[avb]

I stumble over the talk mentioned above just a few days ago, so I'm fairly new to this whole issue... So my question is, why will it detach in this case: QList<MyStruct> &constList = foo(); when it won't detach in this case: const QList<MyStruct> &constList = list; ?

[Resurrection]

@avb The talk described it, no? The issue is that internally Qt containers are COW or shared data. So whenever you do something that could potentially change that data your object will "detach" (make deep copy) of that shared data. This is "hidden" when you return by value (Qt does it a lot because it is cheap because that does not detach on itself). However when you instantiate a non-const iterator on such container it will imediately detach (perform deep copy). const& is fine as it cannot do that byt & can (and is not valid anyway as you cannot take reference to temporary).

[avb]

Yes, I already got that from the talk. But where detaches the container here? Let's say we do this (which is a little different from what I posted before): const QList<MyStruct> constList = foo(); So now we have const Container and no detach so far. Is that right? And now we pass the const Container to the for loop for(const MyStruct &item : constList). Since the Container is const it will use const iterators and also no detach. Or getting here something wrong?

[Resurrection]

@avb That is correct. But if you copied the constList to another non-const list and iterated over that then it would detach again.

[avb]

Ok I get that. But when would a copy from the constList to a non-const list happen in the scenario I described above? I don't see where this should happen!?!

[Resurrection]

It turns out that one needs two overloads of the as_const, one taking l-value (T&) reference (as given) and the other for r-value (T&&) references (as is actually required in the example). Both have same return value and content of course.

[Yakk - Adam Nevraumont]

@Resurrection oops. But note it can be done with one overload as above edit.

[Resurrection]

I have encountered a problem. When a function returns temporary container and it is passed through as_const it gets destroyed after the first element is evaluated instead of lasting for the entire range-loop. I believe the culprit is the as_const function that prolongs the lifetime of the container but only until it itself finishes because it returns only const reference instead of the object and standard defines lifetime of rvalue bound to const ref to be "until expression evaluates" which is as_const in this case and not the loop it seems. Any idea how to solve it?

[Piotr Skotnicki]

Is returning T const deliberate? If so, what's the benefit?

[Yakk - Adam Nevraumont]

@piotr reference lifetime extension wants a value to extend its life. as_const should return a const object, for many reasons (not the least the principle of least surprise). So, it returns a T const - a valur, and const. This also means then auto&& that binds in the for(:) loop generated code will be T const&&, and the const overloads of begin will be called, which is what the OP wanted.

[Resurrection]

The reason I want it for temporaries as well is was for completeness sake on one hand and because of implicit sharing (COW) in Qt on the other. Basically making shallow copies is fast and cheap but the moment you spawn non-const iterator you perform deep copy. Qt classes often return containers by value because it is cheap. But if you iterate over them in non-const way you do the deep copy. If you don't need it it would be a waste (and sometimes significant) to do a deep copy to perform const for-range loop... But maybe I understand it wrongly. :-) Thanks a lot for the forward trick anyhow!

[Yakk - Adam Nevraumont]

@Resurrection Good point; this is a problem with Qt's design (it treats the constness of the view object as the constness of the held data: like treating T const* and T* const the same). But I guess you get what you pay for. :) I have run into the same mistake when I wrote my first views, and ran into similar problems and stopped doing it. The right way to distinguish between views of const and non-const data is to have different types, not const-qualification of the view. A T&const is not a T const&, nor should it be.

[Thomas McGuire]

Newer Qt versions will have qAsConst, see doc-snapshots.qt.io/qt5-dev/qtglobal.html#qAsConst.

[cbuchart]

Just for completeness, std::as_const was introduced in C++17 and it is equivalent to qAsConst.