Extending Python with C/C++

Question

Can anyone please give me tips on the tools or sofware to use to extend Python with C/C++? Thanks.

Solution 1

The Python website itself has a great set of examples, as well as API documentation. That's literally all I used when I needed to write C extensions.

Solution 2

I'll add the obligatory reference to Boost.Python for C++ stuff.

Solution 3

We use SWIG to wrap our C/C++ libraries for use in Python. It works quite well.

http://www.swig.org/

Solution 4

I see nobody's yet pointed out one of my favorite solutions for wrapping C++ code, SIP (I believe it also works for wrapping C, like SWIG and unlike Boost, but I've never used it that way). It's the tool Riverbank Software developed to make PyQt, the Python interface to the wonderful Qt C++ cross-platform framework -- so it's a natural choice if your C++ code uses any Qt functionality, just like Boost Python is the natural choice if your C++ code uses Boost.

SWIG is what we use at work (a reasonable decision when it was made 10 years ago;-) and has the theoretical advantage that it can also wrap C or C++ code for use from Java, Perl, Tcl, etc -- but if you only care about Python it's hard to see anything to make it stand out.

If you're just wrapping an existing DLL/so, besides Cython, which other answers have pointed out (and I endorse, but -- it's changing very fast these days, so take care if you need something more stable), consider the standard function module ctypes -- I wouldn't use it for very extensive work ("oops" errors that a C or C++ compiler would point out to you can cause runtime crashes with ctypes), but for small jobs it's great (and very handy since it comes with standard Python distributions!-).

The good old C API ain't dead yet - just met today with Case, the great guy who's been doing most of the running lately for my good old open source project gmpy, and together we decided to stick with the C API for at least the next release of gmpy -- we'll consider switching to Cython when it stabilizes, but we agreed that the switch would still be a bit premature now. (We didn't even think of any other alternative because gmpy's main point is to be as blindingly fast as we can possibly make it!-).

Solution 5

There are many solutions. In general, you should avoid it if possible, as writing C extensions is tedious. Often, it is necessary to use a 3rd party library. In that case, I think the winning solution today is cython.

Cython is a languages which "looks like python", but can be made much faster by using optional typing. You can call directly C functions inside, and most of the reference counting (the hard problem in C extensions) is done automatically. In my experience, it is much better than boost.python, swig, or ctypes:

• boost.python only makes sense for wrapping C++ extensions IMHO. I find it too complicated, and hard to debug when something goes wrong.
• swig has quite some overhead, and does not lead to good code. It works ok for a couple of functions, but non trivial extensions often use typemaps, and the syntax get ugly quickly

With cython, you can use python objects (list, dict, etc...) to wrap your C library. Of course, it is also very useful if you need to write your own extension just for speed reasons. In the scientific python community, I think cython has become the tool of choice when speed is needed.

View more solutions

Comments

[Exectron]

What do you think about Siboken, the equivalent for PySide? pyside.org/docs/shiboken

[Alex Martelli]

@Craig, haven't tried it yet, I plan to when I next need to do some Qt work (so giving me a realistic chance to try PySide).

[Exectron]

Sorry about the typo, I meant "Shiboken". PySide sounds promising; I'm just sorry they're having to reinvent the wheel.

[andrewrk]

oh I thought you meant figuratively, thanks for the clarification

[Jeffrey Jose]

I started using boost.python few days ago and I'm already up and running.

[Lothar]

Hand written code is the best if the libary is not extraordinary large (like QT or WxWidgets).

[mwag]

Saying "you should avoid it if possible" without knowing the reasons the OP wants to do it in the first place is misguided. There is a reason Python supports C extensions, and it is because sometimes a C extension is the best solution, in which case it should not be avoided. Also ctypes has a lot of advantages over cython depending on context, without which it's misguided to say one is better (see stackoverflow.com/questions/1942298/…)