python - how to reload a cython module interactively using pyximport

Question

When writing python code, my typical workflow is to use the interactive prompt and do something like

write function
repeat until working:
  test function
  edit function

Once I'm pretty sure everything's ok, I'll run the code in non-interactive mode and collect the results.

Sometimes the functions run a little too slow and must be optimized.

I'm interested in using cython to optimize these slow functions, but I want to keep my interactive workflow, i.e., run the functions, make changes, run them again.

Is there an easy way to do this?

So far I've tried putting my cython functions in a separate module "my_functions.pyx":

def fun1(int x):
    return x + 130

def fun2(int x):
    return x / 30

Then running (at the interative prompt)

import pyximport; pyximport.install()
import my_functions as mf
mf.fun1(25)

This works the first time, but I want to make changes to my cython functions and reload them in the same interactive session.

running

import my_functions as mf

doesn't update the functions at all. And running

reload(mf)

gives an error: No module named my_functions

The only thing that works is to quit the current session, restart ipython, and import the module all over again. But this sort of kills the benefits of running interactively.

Is there a better way to optimize functions with cython interactively?

If not, can you describe some other ways to approach optimizing code with cython?

Any help is appreciated.

Answer 1

I found a poorly documented feature in the "pyximport.install" function that allows a cython module to be reloaded. With this feature set to True, you can load/reload your cython modules interactively without having to restart python.

If you initialize your cython module with:

import pyximport
pyximport.install(reload_support=True)
import my_functions as mf

You can make changes to your cython module, and then reload with:

reload(mf)

Hopefully this will be of use to someone.