Static Compiling with Cython

The strength of Numba is the effortless application of the approach to arbitrary functions.

However, Numba will only “effortlessly” generate significant performance improvements

for certain types of problems. Another approach, which is more flexible but also more

involved, is to go the route of static compiling with Cython. In effect, Cython is a hybrid

language of Python and C. Coming from Python, the major differences to be noticed are

the static type declarations (as in C) and a separate compiling step (as with any compiled

language).

As a simple example function, consider the following nested loop that again returns

simply the number of loops. Compared to the previous nested loop example, this time the

number of inner loop iterations is scaled by the outer loop iterations. In such a case, you

will pretty quickly run into memory troubles when you try to apply NumPy for a speedup:

In  [ 80 ]: def f_py(I, J):

res =   0. #    we  work    on  a   float   object

for i in range(I):

for j in range  (J  *   I):

res +=   1

return res

Let us check Python performance for I = 500 and J = 500. A NumPy ndarray object

allowing us to vectorize the function f_py in such a case would already have to have a

shape of (500, 250000):

In  [ 81 ]: I,  J   =    500 ,   500

                                    %time f_py(I,   J)

Out[81]:    CPU times:  user    17  s,  sys:    2.72    s,  total:  19.7    s

                                    Wall    time:   14.2    s

125000000.0

Consider next the code shown in Example 8-1. It takes the very same function and

introduces static type declarations for use with Cython. Note that the suffix of this Cython

file is .pyx.

Example 8-1. Nested loop example with Cython static type declarations

Nested loop example with Cython

nested_loop.pyx

def f_cy(int I, int J):
cdef double res = 0

double float much slower than int or long

for i in range(I):
for j in range (J * I):
res += 1
return res

In such a simple case, when no special C modules are needed, there is an easy way to

import such a module — namely, via pyximport:

In  [ 82 ]: import pyximport

pyximport.install()

Out[82]:    (None,  <pyximport.pyximport.PyxImporter    at  0x92cfc10>)

This allows us now to directly import from the Cython module:

In  [ 83 ]: import sys

sys.path.append(‘data/’)

#   path    to  the Cython  script

#   not needed  if  in  same    directory