Integrate Python And C++

Question

I'm learning C++ because it's a very flexible language. But for internet things like Twitter, Facebook, Delicious and others, Python seems a much better solution.

Is it possible to integrate C++ and Python in the same project?

Solution 1

Interfacing Python with C/C++ is not an easy task.

Here I copy/paste a previous answer on a previous question for the different methods to write a python extension. Featuring Boost.Python, SWIG, Pybindgen...

• You can write an extension yourself in C or C++ with the Python C-API.

  In a word: don't do that except for learning how to do it. It's very difficult to do it correctly. You will have to increment and decrement references by hand and write a lot of code just to expose one function, with very few benefits.

• Swig:

  pro: you can generate bindings for many scripting languages.

  cons: I don't like the way the parser works. I don't know if they've made some progress but two years ago the C++ parser was quite limited. Most of the time I had to copy/paste my .h headers to add some % characters and to give extra hints to the swig parser.

  I also needed to deal with the Python C-API from time to time for (not so) complicated type conversions.

  I'm not using it anymore.

• Boost.Python:

  pro: It's a very complete library. It allows you to do almost everything that is possible with the C-API, but in C++. I never had to write a C-API code with this library. I also never encountered a bug due to the library. Code for bindings either works like a charm or refuses to compile.

  It's probably one of the best solutions currently available if you already have some C++ library to bind. But if you only have a small C function to rewrite, I would probably try with Cython.

  cons: if you don't have a precompiled Boost.Python library you're going to use Bjam (sort of a replacement of make). I really hate Bjam and its syntax.

  Python libraries created with B.P tend to become obese. It also takes a lot of time to compile them.

• Py++: it's Boost.Python made easy. Py++ uses a C++ parser to read your code and then generates Boost.Python code automatically. You also have a great support from its author (no it's not me ;-) ).

  cons: only the problems due to Boost.Python itself.

  Edit this project looks discontinued. While probably still working it may be better to consider switching.

• Pybindgen:

  It generates the code dealing with the C-API. You can either describe functions and classes in a Python file, or let Pybindgen read your headers and generate bindings automatically (for this it uses pygccxml, a python library wrote by the author of Py++).

  cons: it's a young project, with a smaller team than Boost.Python. There are still some limitations: you cannot expose your own C++ exceptions, you cannot use multiple inheritance for your C++ classes.

  Anyway it's worth trying!

• Pyrex and Cython:

  Here you don't write real C/C++ but a mix between Python and C. This intermediate code will generate a regular Python module.

Edit Jul 22 2013: Now Py++ looks discontinued, I'm now looking for a good alternative. I'm currently experimenting with Cython for my C++ library. This language is a mix between Python and C. Within a Cython function you can use either Python or C/C++ entities (functions, variables, objects, ...).

Cython is quite easy to learn, has very good performance, and you can even avoid C/C++ completely if you don't have to interface legacy C++ libraries.

However for C++ it comes with some problems. It is less "automagic" than Py++ was, so it's probably better for stable C++ API (which is now the case of my library). The biggest problem I see with Cython is with C++ polymorphism. With Py++/boost:python I was able to define a virtual method in C++, override it in Python, and have the Python version called within C++. With Cython it's still doable but you need to explicitly use the C-Python API.

Edit 2017-10-06:

There is a new one, pybind11, similar to Boost.Python but with some potential advantages. For example it uses C++11 language features to make it simpler to create new bindings. Also it is a header-only library, so there is nothing to compile before using it, and no library to link.

I played with it a little bit and it was indeed quite simple and pleasant to use. My only fear is that like Boot.Python it could lead to long compilation time and large libraries. I haven't done any benchmark yet.

Solution 2

Yes, it is possible, encouraged and documented. I have done it myself and found it to be very easy.

Solution 3

We use swig very successfully in our product.

Basically swig takes your C++ code and generates a python wrapper around it.

Solution 4

Python/C API Reference Manual - the API used by C and C++ programmers who want to write extension modules or embed Python.

Extending and Embedding the Python Interpreter

describes how to write modules in C or C++ to extend the Python interpreter with new modules. Those modules can define new functions but also new object types and their methods. The document also describes how to embed the Python interpreter in another application, for use as an extension language. Finally, it shows how to compile and link extension modules so that they can be loaded dynamically (at run time) into the interpreter, if the underlying operating system supports this feature.

Solution 5

Try Pyrex. Makes writing C++ extensions for Python easier.

View more solutions

Comments

[Nathan Campos]

Thanks for the experience and the very nice documentation!

[Matt Price]

Also check out the doc OttoA posted: docs.python.org/extending/extending.html

[Web_Geeek2018]

Python != CPython. CPython is written in C. C != C++.

[kabdulla]

Python is written in C and not C++. That's why many nice C++ features (like automatic reference counting) must be done by hand if you use the Python C-API.

[kabdulla]

How many functions/classes dit you expose to Python to find this task easy ?

[Mathieu]

@ascobol: you get the hang of it pretty quickly.

[Zoran Pavlovic]

Note, your link to Py++ is no longer working. I think they moved to here: ohloh.net/p/pygccxml

[kabdulla]

@ZoranPavlovic: indeed Py++ looks discontinued, I updated the link. Thanks

[kirbyfan64sos]

You should add Shiboken, the one used for PySide. Horribly documented, but awesome.

[Kaushik Ghose]

Cython is the successor to Pyrex. (Leaving comment to help those visiting this answer in 2014)

[Brian]

I think it would be more accurate to call Cython a fork of Pyrex. For several years, Pyrex and Cython were in simultaneous development. Mind you, at this point Pyrex is no longer under active development and thus not an ideal option for new development.

[SexyBeast]

Hi, I went to the documentation. What are the libraries that need to be included to be able to call the Python functions? Where do I get the Python.h from?

[Wenzel Jakob]

FYI: The pybind11 documentation includes a benchmark with regards to compilation time and code size (pybind11.readthedocs.org/en/latest/benchmark.html)

[lads]

@ascobol do you know any good(easy to use) non open source libraries ?

[Munick]

To save the next person a few clicks and a google, they're using cffi and have clean, simple custom utils and a repo with examples on how to use their utils to extend pytorch.