Is there a way to run C++ Unit Tests tests in parallel?

Question

I'm using Boost Test for a long time now and I ends up having my tests running too slowly. As each test is highly parallel, I want them to run concurrently with all my cores.

Is there a way to do that using the Boost Test Library ? I didn't found any solution. I tried to look a how to write custom test runner, but I didn't much documentation on that point :(

If there is no way, does someone know a good C++ Test Framework to achieve that goal ? I was thinking that Google Test would do the job but apparently it cannot run test in parallel either. Even if the framework has less features than other more known framework, it is not a problem, I just need simple assertions and multi-threaded execution.

Thanks

Solution 1

You could use CTest for this.

CTest is the test driver which accompanies CMake (the build system generator), so you'd need to use CMake to create the build system using your existing files and tests, and in doing so you would then be able to use CTest to run the test executables.

I haven't personally used Boost.Test with CMake (we use GoogleTest), but this question goes into a little more detail on the process.

Once you have the tests added in your CMakeLists file, you can make use of CTest's -j argument to specify how many jobs to run in parallel.

Solution 2

What google is hinting at in the gtest documentation is test sharding - letting multiple machines run the tests by just using command line parameters and environment variables. You could run them all on one machines in separated processes, where you set the GTEST_SHARD_INDEX and GTEST_TOTAL_SHARDS environment variables appropriately.

In principle, nothing is preventing you from starting multiple processes of the test executable with a different filtering parameter (Boost.test, gtest)

Update 2014: https://github.com/google/gtest-parallel

Solution 3

Split the suite to consist of multiple smaller sets each launched with individual binary, and add .PHONY target test to your build sysem depending on all of them. Run as (assuming you are using make) make -jN test

Comments

[Baptiste Wicht]

That is a good idea, but the problem with that is that I have to create as many executable test as I have tests, I prefer something more automated.

[Baptiste Wicht]

That is awesome :) Even if it is a bit messy and I had to make some changes to the cmake files, it works ! Thanks a lot

[bobah]

@BaptisteWicht - you can have a single launcher taking the test name as a command line parameter (like Boost.Test does)