Compile the Python interpreter statically?

Question

I'm building a special-purpose embedded Python interpreter and want to avoid having dependencies on dynamic libraries so I want to compile the interpreter with static libraries instead (e.g. libc.a not libc.so).

I would also like to statically link all dynamic libraries that are part of the Python standard library. I know this can be done using Freeze.py, but is there an alternative so that it can be done in one step?

Solution 1

I found this (mainly concerning static compilation of Python modules):

• http://bytes.com/groups/python/23235-build-static-python-executable-linux

Which describes a file used for configuration located here:

<Python_Source>/Modules/Setup

If this file isn't present, it can be created by copying:

<Python_Source>/Modules/Setup.dist

The Setup file has tons of documentation in it and the README included with the source offers lots of good compilation information as well.

I haven't tried compiling yet, but I think with these resources, I should be successful when I try. I will post my results as a comment here.

Update

To get a pure-static python executable, you must also configure as follows:

./configure LDFLAGS="-static -static-libgcc" CPPFLAGS="-static"

Once you build with these flags enabled, you will likely get lots of warnings about "renaming because library isn't present". This means that you have not configured Modules/Setup correctly and need to:

a) add a single line (near the top) like this:

*static*

(that's asterisk/star the word "static" and asterisk with no spaces)

b) uncomment all modules that you want to be available statically (such as math, array, etc...)

You may also need to add specific linker flags (as mentioned in the link I posted above). My experience so far has been that the libraries are working without modification.

It may also be helpful to run make with as follows:

make 2>&1 | grep 'renaming'

This will show all modules that are failing to compile due to being statically linked.

Solution 2

CPython CMake Buildsystem offers an alternative way to build Python, using CMake.

It can build python lib statically, and include in that lib all the modules you want. Just set CMake's options

BUILD_SHARED                     OFF
BUILD_STATIC                     ON

and set the BUILTIN_<extension> you want to ON.

Solution 3

Using freeze doesn't prevent doing it all in one run (no matter what approach you use, you will need multiple build steps - e.g. many compiler invocations). First, you edit Modules/Setup to include all extension modules that you want. Next, you build Python, getting libpythonxy.a. Then, you run freeze, getting a number of C files and a config.c. You compile these as well, and integrate them into libpythonxy.a (or create a separate library).

You do all this once, for each architecture and Python version you want to integrate. When building your application, you only link with libpythonxy.a, and the library that freeze has produced.

Solution 4

You can try with ELF STATIFIER. I've been used it before and it works fairly well. I just had problems with it in a couple of cases and then I had to use another similar program called Ermine. Unfortunately this one is a commercial program.

View more solutions

Comments

[Jeremy Cowles]

I'm always suspicious when someone says "never blah blah blah"; in my scenario, it completely made sense to statically link. Thanks for the link though.

[0xC0000022L]

@sehe: there are many good reasons for static linking, especially with proprietary software. Good for the folks who earn their living with FLOSS, I still work for a proprietary shop and need to consider the needs of my employer. Tainting code with GPL or LGPL code is out of the question for us. Static linking and getting rid of glibc and friends is one way to achieve that.

[sehe]

I appreciate that static linkage has a purpose. However, it's relation to Licensing is ... tenuous at best. If licensing is going to be the argument, then surely dynamic linking trumps static linking by a big margin, because dynamic linking is not considered a derivative work in the sense of GPL, but static linking is. IANAL though. Anyways, static linking is not a way to get rid of glibc. Not using glibc (static or dynamic) is the only way to achieve that

[sehe]

@Navin Interesting. I wonder how it would have taken me so many years to finally get someone to explain it in that particular way to me. Do you have a source that specifically makes that goal clear, and relates it to the license wording? I will definitely read up on that.

[Jeremy Cowles]

Thanks for the response, but you glossed over the part that I needed: how do you "build Python getting libpythonxy.a"? Can you give a little more detail? Or is it as simple as just running make and the lib will be generated automatically?

[Jeremy Cowles]

Thanks for the help, but I don't want a pseudo-static library, I want to link statically.

[Seng Cheong]

Shouldn't that be make 2>&1

[thodnev]

If there are some issues while building PyDateTime, use this hack bugs.python.org/issue19348

[Noah]

To get ssl module working you may need to add -ldl