"Mixing a dll boost library with a static runtime is a really bad idea..."

Question

I have two projects in a Visual Studio solution. One builds a static LIB, the other builds a dynamic DLL. Both use static runtime linking (/MT and /MTd), and both use Boost. Boost was not my decision - I wanted to chuck it but I was overruled by committee.

The LIB builds fine, but the DLL coughs up an error from auto_link.hpp (line 354): "Mixing a dll boost library with a static runtime is a really bad idea...".

#if (defined(_DLL) || defined(_RTLDLL)) && defined(BOOST_DYN_LINK)
#  define BOOST_LIB_PREFIX
#elif defined(BOOST_DYN_LINK)
#  error "Mixing a dll boost library with a static runtime is a really bad idea..."
#else
#  define BOOST_LIB_PREFIX "lib"
#endif

I did not define BOOST_DYN_LINK. It seems Boost is making a leap that since I am building a DLL (_USRDLL and _WINDLL are defined), I must want dynamic runtime linking (/MD or /MDd, which defines _DLL) or DLL linking against Boost. This is incorrect as I specifically asked for static linking (/MT or /MTd).

I filed a bug report against Boost for its incorrect assumptions, but that does not help me with using the library. In the report, the Boost maintainers insist that I am setting it (despite the fact that an audit showed I am not; and Boost manipulates it in at least 30 files). I found one answer on the Boost mailing list, which essentially states to change my project settings to accomodate Boost.

Changing to Dynamic Runtime Linking (/MD and /MDd) is not feasible since static linking was chosen (1) due to security considerations, and (2) another library uses static linking. This is non-negotiable - we have no choice.

To summarize for clarity (TLDR): I want to use static linking for everything, while my output program is a DLL (not a static LIB, not an EXE). Everything is linked statically within the DLL.

Does anyone know how to use this library on Windows to build a DLL with static linking?

Solution 1

This issue is indeed a fault with boosts settings. For some unknown reason (that I cannot determine to be logical - as it has not effect). Boost Python will force dynamic linking of boost, regardless of user options

In short, if you have boost-python in your project, boost incorrectly prevents static-linking of boost with the /MT switch.

This problem is resolved easily by defining BOOST_PYTHON_STATIC_LIB before including the boost headers.

Solution 2

Short version: listen to the guys on the Boost ML. Boost doesn't support what you're trying to do, and it's a bad idea anyway. Best to accept that and make the other library use the dynamic runtime.

Longer version:

You seem to be misunderstanding what Boost is telling you. You're trying to parse through the Boost source code instead of just reading what it's saying:

auto_link.hpp (line 354): "Mixing a dll boost library with a static runtime is a really bad idea...".

Boost thinks that you are building a DLL, while simultaneously linking statically to the runtime libraries. Which is exactly what you're doing; Boost has accurately detected what you are trying to do.

Your problem is that the Boost library does not support being built into a DLL that is not dynamically linking to the runtime libraries. The reason for that is that it is "a really bad idea." So they check to see if you're trying to do that and stop your build with an error message reminding you of this fact.

The "freetards" at Boost who don't know how to make something work "out of the box" prevent this because statically linking to the runtime in a DLL is usually a mistake by the user. It's either made accidentally or through ignorance of the major problems this can create.

If you statically link to the runtimes, each DLL/exe will have its own copy of the runtimes, with their own global variables. And since the heap is managed via globals, this means that each DLL/exe will have its own heap. So if you try to free memory allocated in another address space... boom. And this is a lot easier than you think if you're not careful.

This can cause other problems as well. The "freetards" are trying to stop you from shooting yourself in the foot. But obviously you know better than to "force their crap on honest and unsuspecting users" who might want to be advised when they're about to drive over a cliff.

Now, you could simply remove the error message. Most of Boost is headers, so as long as you're not actually linking to any of its .libs, you should be fine. However, I'd guess that "auto_link.hpp" is only used by the parts of Boost that are .libs, odds are good that the fact you encountered it to begin with means you're trying to link to a Boost .lib.

Changing to Dynamic Runtime Linking (/MD and /MDd) is not feasible since static linking was chosen (1) due to security considerations, and (2) another library uses static linking.

If security is a consideration, you should be aware of this: the very fact that you're building a DLL means that your application is potentially open to DLL injections, regardless of how you link to the runtimes. So I don't see how dynamic linking is any less secure than static.

Comments

[Luke]

I don't think this is right. auto_link.hpp is checking for _DLL which is defined based on what CRT you are linking with, not what type of project you are building. boost comes with pre-built versions of its libraries that link with the static CRT; we use them in a DLL just fine. Obviously we do not use the boost classes in the interface of the DLL for the reasons you stated.

[Nicol Bolas]

@Luke: It doesn't matter what the #defines are; the error he's getting exactly matches how he says he's building his DLL. The Boost error message is saying that he can't build a DLL that way.

[Luke]

Of course the defines matter; that's what the error message is based off of. I agree that some configuration option is incorrect somewhere, but it is possible to build in the configuration described in the original question. boost ships binaries in multiple configurations, one of which is boost as a static lib that links with the static CRT. We link this into a DLL without issue.

[Harry Johnston]

Why would there be any problem with the heap? Surely the heap functions in the C runtime library just call the Windows heap functions?

[Nicol Bolas]

@HarryJohnston: The Windows heap functions return blocks of memory in page sizes. So the runtime heap functions have to divide the blocks of memory to your application, managing memory at a finer granularity than 4KB pages.

[Harry Johnston]

@nicol: no, the Windows heap functions don't return page-size blocks. Check the documentation, or try it yourself.

[shf301]

@HarryJohnston When say "the heap" you are making the assumption that there is only one heap. There are multiple heaps and each instance of the C runtime creates its heap - and that's the problem.

[Harry Johnston]

@shf301: I haven't looked into the runtime's code, so I can't say that you're wrong, but why wouldn't it just use the process default heap?

[Harry Johnston]

Anyhoo ... if the odd behaviour of the MS C runtime is a problem this might be a sensible alternative: benshoof.org/blog/minicrt

[jww]

"Boost thinks that you are building a DLL, while simultaneously linking statically to the runtime libraries. Which is exactly what you're doing; Boost has accurately detected what you are trying to do." - I'm not trying to use a Boost DLL. I am building a DLL, and I want to import Boost as a static LIB (just like all the other libraries I am using).

[Nicol Bolas]

@noloader: Neither I nor Boost said you were using Boost as a DLL. You are building a DLL.

[jww]

"dll boost library" - what does that mean to you?

[jww]

"The 'freetards' at Boost who don't know how to make something work 'out of the box' prevent this because statically linking to the runtime in a DLL is usually a mistake by the user." - The security considerations extend well beyond binary planting. This software might be FIPS certified at some point. Our output library must be a DLL to enforce a security boundary. Depending on the testing lab, they may (or may not) insist on static linking.

[Harry Johnston]

@noloader: you've got me curious now. In what sense is a DLL a security boundary? (A link would be an ideal answer.)

[jww]

@Harry - see FIPS 104-2 and FIPS 140-2 and CMVP Implementation Guidance. The DLL is the logical module boundary, and its exported methods are the supported interfaces.

[Mark Ransom]

@noloader, if a DLL is preferred for Boost I don't see why it isn't preferred for the C runtime library. You're setting yourself up for failure by insisting on the setup you've chosen - things are going to go boom for no explainable reason. That's what the error is trying to tell you.

[dothebart]

I ran into the same trap with unit tests; github.com/triAGENS/ArangoDB/blob/master/UnitTests/Basics/… demands dynamic; It seems as if cmake doesn't offer a way to mix static and dynamic linking under windows? It seems the sf.net packages do not provide static, since after I got the warning away by commenting out BOOST_TEST_DYN_LINK in the file it would compile, but fail while linking... Or did I get it wrong completely?

[jww]

I'm going to accept this to help future visitors because it seems like a reasonable answer. But a caveat to future visitors: I was not able to test it.

[bw4sz]

Can you be specific about what you mean as defining BOOST_PYTHON_STATIC_LIB. The absolute path to the library? I'm having this error and can't seem to crack it. Just one line of example code?

[John Bargman]

@bw4sz , Quite before including the boost header #define BOOST_PYTHON_STATIC_LIB , or set it as a preprocessor definition in your IDE/Makefile