python - cython: relative cimport beyond main package is not allowed

Question

I am trying to use explicit relative imports in cython. From the release notes it seems like relative imports should work after cython 0.23, and I'm using 0.23.4 with python 3.5. But I get this strange error that I cannot find many references to. The error is only from the cimport:

driver.pyx:4:0: relative cimport beyond main package is not allowed

The directory structure is:

    myProject/
        setup.py
        __init__.py
        test/
            driver.pyx
            other.pyx
            other.pxd

It seems like I'm probably messing up in setup.py so I included all the files below.

setup.py

from setuptools import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

ext_modules = [
    Extension('other', ['test/other.pyx'],),
    Extension('driver', ['test/driver.pyx'],),
]

setup(
    name='Test',
    ext_modules=ext_modules,
    include_dirs=["test/"],
    cmdclass={'build_ext': build_ext},
)

driver.pyx

#!/usr/bin/env python
from . import other
from . cimport other

other.pyx

#!/usr/bin/env python

HI = "Hello"

cdef class Other:
    def __init__(self):
        self.name = "Test"

    cdef get_name(self):
        return self.name

other.pxd

cdef class Other:
    cdef get_name(self)

I've tried moving __init__.py into test/. I've tried running setup.py in the test directory (adjusting the include_dirs appropriately). They both give the same error.

If I do cimport other and remove the . it works but this is a toy example and I need relative imports so other folders can import properly. This is the only example I can find for this error and I'm pretty confident my issue is different.

Answer 1

The only other example I could find of this error was in hal.pyx of the machinekit project. I was pretty confident that this was a different error but today I realized that after that error was solved machinekit was working which means explicit relative imports must work. Their setup.py file refers to linuxcnc which isn't in the directory tree but I guess is created at some point during compile time. The important thing is that the include_dirs includes the parent directory instead of the child directory.

Translated to my project structure it would mean I put myProject in include_dirs instead of test/. After read this guide for the second time I finally started to understand a little of how python thinks of packages. The problem was that the include_dirs was the child directory. It seems like this effectively made cython view it as a single flat directory in which case no relative imports would be allowed? An error like this might have made it more clear:

ValueError: Attempted relative import in non-package

I still don't have a sufficiently deep understanding to know exactly what was going on but luckily the solution was relatively simple. I just changed the include_dirs to make cython recognize the nested file structure:

from setuptools import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

ext_modules = [
    Extension('other', ['test/other.pyx'],),
    Extension('driver', ['test/driver.pyx'],),
]

setup(
    name='Test',
    ext_modules=ext_modules,
    include_dirs=["."],
    cmdclass={'build_ext': build_ext},
)

It all works now!