How to set the stacksize with C++11 std::thread

Question

I've been trying to familiarize myself with the std::thread library in C++11, and have arrived at a stumbling block.

Initially I come from a posix threads background, and was wondering how does one setup the stack size of the std::thread prior to construction, as I can't seem to find any references to performing such a task.

Using pthreads setting the stack size is done like this:

void* foo(void* arg);
.
.
.
.
pthread_attr_t attribute;
pthread_t thread;

pthread_attr_init(&attribute);
pthread_attr_setstacksize(&attribute,1024);
pthread_create(&thread,&attribute,foo,0);
pthread_join(thread,0);

Is there something similar when using std::thread?

I've been using the following reference:

http://en.cppreference.com/w/cpp/thread

Solution 1

Initially I come from a posix threads background, and was wondering how does one setup the stack size of the std::thread prior to construction, as I can't seem to find any references to performing such a task.

You can't. std::thread doesn't support this because std::thread is standardized, and C++ does not require that a machine even has a stack, much less a fixed-size one.

pthreads are more restrictive in terms of the hardware that they support, and it assumes that there is some fixed stack size per thread. (So you can configure this)

Solution 2

As Loki Astari already said, it is extremely rare to actually need a non-default stack-size and usually either a mistake or the result of bad coding.

• If you feel like the default stack size is too big for your needs and want to reduce it, just forget about it. Every modern OS now uses virtual memory / on-demand commit, which means that memory is only reserved, not actually allocated, until you access the pages. Reducing the stack size will not reduce your actual memory footprint.

• Due to this very behaviour, OSes can afford to set the default stack size to very big values. E.g. on a vanilla Debian this is 8MB (ulimit -s) which should be enough for every need. If you still manage to hit that limit, my first idea would be that your code is wrong, so you should first and foremost review it and move things to the heap, transform recursive functions into loops, etc.

• If despite all of this you really really need to change the stack size (i.e. increase it, since reducing it is useless), on POSIX you can always use setrlimit at the start of your program to increase the default stack size. Sure this will affect all threads, but only the ones who need it will actually use the additional memory.

• Last but not least, in all fairness I can see a corner case where reducing the stack size would make sense: if you have tons of threads on a 32 bits system, they could eat up your virtual address space (again, not the actual memory consumption) up to the point that you don't have enough address space available for the heap. Again, setrlimit is your friend here even though I'd advise to move to a 64 bits system to benefit from the larger virtual address space (and if your program is that big anyway, you'll probably benefit from the additional RAM too).

Solution 3

I have also been investigating this issue. For some applications, the default stack size is not adequate. Examples: the program does deep recursion dependent on the specific problem it is solving; the program needs to create many threads and memory consumption is an issue.

Here is a summary of (partial) solutions / workarounds I found:

• g++ supports a -fsplit-stack option on Linux. See for more information about Split stacks. Here is summary from their website:

The goal of split stacks is to permit a discontiguous stack which is grown automatically as needed. This means that you can run multiple threads, each starting with a small stack, and have the stack grow and shrink as required by the program.

Remark: -fsplit-stack only worked for me after I started using the gold linker. It seems clang++ will also support this flag. The version I tried (clang++ 3.3) crashed when trying to compile my application using the flag -fsplit-stack.

• On Linux, set the stack size by executing ulimit -s <size> before starting your application. size is the stack size in Kbs. Remark: the command unlimit -s unlimited did not affect the size of threads created with std::thread. When I used ulimit -s unlimited, the main thread could grow, but the threads created with std::thread had the default size.

• On Windows using Visual Studio, we can use use the linker /STACK parameter or /STACKSIZE in the module definition file, this is the default size for all created threads. See this link for more information. We can also modify this parameter in any executable using the command line tool EDITBIN.

• On Windows using mingw g++, we can use the option -Wl,--stack,<size>. For some reason, when using cygwin g++, this flag only affects the size of the main thread.

Approaches that did not work for me:

• ulimit -s <size> on OSX. It affects only the size of the main thread. Moreover, the Mac OSX default for a pthread stack size is 512kB.

• setrlimit only affects the size of the main thread on Linux and OSX. On cygwin, it never worked for me, it seems it always returns an error.

For OSX, the only alternative seems to use boost::thread instead of std::thread, but this is not nice if we want to stick with the standard. I hope g++ and clang++ will also support -fsplit-stack on OSX in the future.

Solution 4

I found this in Scott Meyers book Overview of the New C++(C++0x), as it's quite long I can't post it as a comment, is this helpful?

There is also a standard API for getting at the platform-specific handles behind threads, mutexes, condition variables, etc.. These handles are assumed to be the mechanism for setting thread priorities, setting stack sizes, etc. (Regarding setting stack sizes, Anthony Williams notes: "Of those OSs that support setting the stack size, they all do it differently. If you're coding for a specify platform (such that use of the native_handle would be OK), then you could use that platform's facilities to switch stacks. e.g. on POSIX you could use makecontext and swapcontext along with explicit allocation of a stack, and on Windows you could use Fibers. You could then use the platform-specific facilities (e.g. Linker flags) to set the default stack size to something really tiny, and then switch stacks to something bigger where necessary.“)

Solution 5

Was looking for the answer to this myself just now.

It appears that while std::thread does not support this, boost::thread does.

In particular, you can use boost::thread::attributes to accomplish this:

boost::thread::attributes attrs;
attrs.set_stack_size(4096*10);
boost::thread myThread(attrs, fooFunction, 42);

View more solutions

Comments

[billz]

you could get native handler and set stack size? en.cppreference.com/w/cpp/thread/thread/native_handle

[Zamfir Kerlukson]

@billz Once the thread has been created it is too late.

[Billy ONeal]

Why do you want to do this?

[Zamfir Kerlukson]

@Billy: There are many scenarios where one may want to create many threads for a thread pool and assume that all the data/memory the thread will require will external hence the stack can be set small - essentially enough to house any calls or code nothing else.

[hanshenrik]

@BillyONeal i'm working on a program that needs a lot worker threads, these threads doesn't need a big stack size (running them at a 500 kb stack/worker thread), the std::thread default stack size is 8 megabytes/thread. if i want 10 worker threads, that's either 80 megabytes of ram using std::thread, or 5 megabytes of ram using pthreads..

[Billy ONeal]

@hanshenrik: Not really. 8MB (on Linux) and 1MB (on Windows) is the default stack size reservation -- it'll only generally consume physical RAM if pages are touched.

[hanshenrik]

@BillyONeal so you're saying a thread that doesn't do anything, just void threadEntry(){for(;;){this_thread::sleep_for(seconds(1));}} but have a stack size of 0.5MB use just as much memory as a thread doing the same with a 8MB stack size?

[hanshenrik]

@BillyONeal damn, seems you're right, i wrote some code to test it, then installed a debian 9 amd64 minimal with kernel 4.9.110 , g++ 6.3.0, and 152 MB ram and 0 swap (which seems to be nearly the minimum debian will boot on without tweaking the initial ramdisk - somewhere between 140 and 152MB ram) on VMWare, and both mode 1 (std::thread with 8MB stack) and mode 2 (pthread with 512 kb stack) are able to run exactly 700 threads before failing! cool - during boot it needs ~152 MB but after boot its 65MB ram, unfortunate

[Zamfir Kerlukson]

+1 but that is a very disappointing and short-sighted result. I'd have expected a trait or stack allocator trait mechanism at the very least.

[Zamfir Kerlukson]

+1 Thats an interesting albeit round about way of setting the stacksize.

[Martin York]

@ZamfirKerlukson: The language designers did think about this. You get the native handle from the Thread object. Then manipulate it with appropriate native function calls for your platform. But the reason they made it hard is that doing it is usually a mistake and when actually required is very/very rare.

[Leonardo de Moura]

I tried setrlimit on Linux and Cygwin. On Cygwin, it always fails. On Linux, it only changes the size of the main thread. The ulimit -s solution works on Linux. On OSX, it only affects the size of the main thread. The Mac OS X default for a pthread stack size is 512kB. This is quite small. It is a pity we can't change the default stack size using std::thread.

[Leonardo de Moura]

I don't see how to change the stack size using the native handle. Any hints? Moreover, the Mac OSX default for a pthread stack size is 512kB. This is quite small.

[Elazar Leibovich]

Doesn't help you if you want to configure it before the thread starts.

[user1095108]

640 kb used to be enough for everybody

[Billy ONeal]

As far as I am aware you can't change the stack size for a thread after it has started.

[Admin]

", it is extremely rare to actually need a non-default stack-size". Speak for yourself. I work with large matrix operations and often the default stack size is not enough.

[Sebastian]

On microcontrollers you most likely want to be able to set the stack size. Just another agument to support the preconception that C++ is not good for microcontrollers. Sad.

[Rich von Lehe]

@Sebastian I agree that being able to control stack sizes on a microcontroller would be helpful, but I feel it's more common to be using an RTOS in those environments in which case you'd be using that RTOS's API for thread manipulation. C++ is still viable there, albeit with some features being less useful.

[Martin Kosicky]

I like so generic software that it has 0 flexibility. I wonder if there exists an implementation that has no stack?

[Alex Reinking]

Windows has a 1MB default stack. Clang has to increase the stack size to have enough space to instantiate recursive templates. Are they writing bad code?