Resolving circular dependencies by linking the same library twice?
Solution 1
All I can offer is a lack of counter-example. I've actually never seen the first form before (even though it's clearly better) and always seen this solved with the second form, and haven't observed problems as a result.
Even so I would still suggest changing to the first form because it clearly shows the relationship between the libraries rather than relying on the linker behaving in a particular way.
That said, I would suggest at least considering if there's a possibility of refactoring the code to pull out the common pieces into additional libraries.
Solution 2
The problem with
g++ -o myApp -lfoo -lbar -lfoo
is that there is no guarantee, that two passes over libfoo
and one pass over libbar
are enough.
The approach with Wl,--start-group ... -Wl,--end-group
is better, because more robust.
Consider the following scenario (all symbols are in different object-files):
myApp
needs symbolfooA
defined inlibfoo
.- Symbol
fooA
needs symbolbarB
defined inlibbar
. - Symbol
barB
needs symbolfooC
defined inlibfoo
. This is the circular dependency, which can be handled by-lfoo -lbar -lfoo
. - Symbol
fooC
needs symbolbarD
defined inlibbar
.
To be able to build in the case above, we would need to pass -lfoo -lbar -lfoo -lbar
to the linker. Why?
- The linker sees
libfoo
for the first time and uses definitions of symbolfooA
but notfooC
, because so far it doesn't see a necessity to include alsofooC
into the binary. The linker however starts to look for definition ofbarB
, because its definition is needed forfooA
to function. - The linker sees
-libbar
, includes the definition ofbarB
(but notbarD
) and starts to look for definition offooC
. - The definition of
fooC
is found inlibfoo
, when it processed for the second time. Now it becomes evident, that also the definition ofbarD
is needed - but too late there is nolibbar
on the command line anymore!
The example above can be extended to an arbitrary dependency depth (but this happens seldom in real life).
Thus using
g++ -o myApp -Wl,--start-group -lfoo -lbar -Wl,--end-group
is a more robust approach, because linker passes as often over the library group as needed - only when a pass didn't change the symbol table will the linker move on to the the next library on the command line.
There is however a small performance penalty to pay: in the first example -lbar
were scanned once more compared with the manual command line -lfoo -lbar -lfoo
. Not sure whether it is worth mentioning/thinking about through.
Solution 3
Since it is a legacy application, I bet the structure of the libraries is inherited from some arrangement which probably does not matter any more, such as being used to build another product which you no longer do.
Even if still structural reasons remain for the inherited library structure, almost certainly, it would still be acceptable to build one more library from the legacy arrangement. Just put all the modules from the 20 libraries into a new library, liballofthem.a
. Then every single application is simply g++ -o myApp -lallofthem ...
Nemo
Updated on June 13, 2022Comments
-
Nemo almost 2 years
We have a code base broken up into static libraries. Unfortunately, the libraries have circular dependencies; e.g.,
libfoo.a
depends onlibbar.a
and vice-versa.I know the "correct" way to handle this is to use the linker's
--start-group
and--end-group
options, like so:g++ -o myApp -Wl,--start-group -lfoo -lbar -Wl,--end-group
But in our existing Makefiles, the problem is typically handled like this:
g++ -o myApp -lfoo -lbar -lfoo
(Imagine this extended to ~20 libraries with complex interdependencies.)
I have been going through our Makefiles changing the second form to the first, but now my co-workers are asking me why... And other than "because it's cleaner" and a vague sense that the other form is risky, I do not have a good answer.
So, can linking the same library multiple times ever create a problem? For example, could the link fail with multiply-defined symbols if the same .o gets pulled in twice? Or is there any risk we could wind up with two copies of the same static object, creating subtle bugs?
Basically, I want to know if there is any possibility of link-time or run-time failures from linking the same library multiple times; and if so, how to trigger them. Thanks.
-
Nemo about 12 yearsThanks, Mark. Although I do find it amusing that half of the comments on my question say "Fix your codebase!" and the other half say "Why are you tampering with a working codebase?" :-)
-
ed9w2in6 over 11 yearsThe first form would introduce a performance cost as the linker try to find symbols repeated across all listed libraries. see this: stackoverflow.com/a/409470/70198
-
user1225999 over 9 yearsThe first form is also only working for GNU ld and is thus not a portable solution.
-
Grim Fandango about 3 yearsis the
--start-group ... --end-group
just a convenience switch, or does it deal with the circular dependencies faster, by cacheing the .o files or the symbol tables of the whole group? I mean, if I turn theg++ -o myApp -lfoo -lbar -lfoo -lbar -lfoo
into a--start-group ... --end-group
do I expect the linker to scan thelfoo
andlbar
archives just once each? -
FallenWarrior about 3 yearsIt's not a convenience switch because it will continue to work, even if more cycles are added in the code. The handwritten command line will break if barD suddenly needs fooE, even though dependency-wise, "nothing" has changed, as it's still foo and bar depending on each other. To clarify, simply pulling in additional symbols from libraries you already link against might break the build.