Python/NumPy performance

This post presents some of my thoughts about trying several methods to “bind” C code with Python, in particular to interact with NumPy arrays. It is meant to be modified over time, and is not intended to be a tutorial. With time, there should be enough links and remarks to help whoever tries and stumbles, as anyone, on the path of using open source solutions, the “free-ness” of which comes at the cost of – generally – insufficient documentation. Comments are welcome!

Python/Scipy and scipy.weave.inline

This was my preferred solution until recently. The good: take your C/C++ code, load it as a string, and use it as support code for the scipy.weave.inline. With the right call, inline can convert the np.ndarray into the desired C arrays.
Unfortunately, this simplicity comes at the cost of the organization of the files: the compiled extension is done in some “obscure” place, in your folders, where Python can find them, with very unlikely names (not related to your code). This renders it difficult to use afterwards, in addition to the fact that one never really knows when it is compiled again (except by setting the force option to 1, but this is in some cases unsatisfying: if you need to call that function several times, in an algorithm, then it will compile it each time…).

Python/NumPy with Cython

This seems to be the preferred method adopted by the numpy and scipy (and scikits.learn, by the way), recently.
In short, and from a user perspective: it allows you to write a python program, and when needed add keywords here and there, which are going to be processed by the Cython wrapper.

Python/NumPy with SWIG

SWIG is yet another solution, which has been dropped by the devs of numpy, but for which I recently gave a look.

A few things to pay attention to before using SWIG:

Remarks and issues addressed (or not) with using SWIG and Python, especially with Python/NumPy

	Issues	Solutions
name of module	The name of the module is somewhat… difficult to apprehend, especially since it’s not really explained in the doc! The error I got was: ImportError: dynamic module does not define init function (initexample)	When the module is called, it seems that in the inteface file, example.i, the module name should be example, while in distutils’ setup.py, the module name in the Extension instantiation should be _example. Not sure where this is explicitly said, but that’s how it worked for me. Reference: swig doc1
Use of NumPy arrays	Of course, the use of NumPy arrays needs special care, because	Use of typemaps, thanks to the file provided in NumPy (older versions?)2
OMP	20120614T1338 : Use of Open MP in SWIG seems to be an issue with MacOSX 10.6. Somehow, the program hangs at some point. With activity monitor, inspecting the samples of the process, it seems to stop at a call to __spin_lock (whatever that may be… )	No solution! Well, a reboot, a fresh build (python setup.py build_ext –inplace), and it now seems to work. But since the issue appear at random, I may expect it again… ô joie.