python - cython reuse wrapped C++ class

Question

I would like to wrap some C++ class with cython, but my problem is that I have to use some class in another classes. I have not find anyone else with the same problem, not even a similar one so sorry if it is already asked... I made a minimalistic example of it which is already pretty long...

So I have the following file structure, where BuildAll.sh run through the folders, and if it has a BuildAll.sh file it'll go deeper, and if a folder has a setup.py file it'll build it (the setup.py file will look for every *.pyx file and build them) Now for dummy purposes classAC is the ClassA written in C++, which holds an int as a member. The same is in the B folder, but classBC holds an instance of classAC as a member and I think this is the problem since classBC don't know classACs definitions.

.
├── BuildAll.sh
├── main.py
└── src
    ├── A
    │?? ├── classAC.cpp
    │?? ├── classAC.hpp
    │?? ├── classAC.pxd
    │?? ├── classA.pyx
    │?? ├── __init__.py
    │?? └── setup.py
    ├── B
    │?? ├── classBC.cpp
    │?? ├── classBC.hpp
    │?? ├── classBC.pxd
    │?? ├── classB.pyx
    │?? ├── __init__.py
    │?? └── setup.py
    ├── BuildAll.sh
    └── __init__.py

BuildAll.sh:

#!/bin/bash

# Go into every directory
for D in */
do
    cd $D
    # If dir contains a BuildAll.sh script, run it.
    if test -f "BuildAll.sh"; then 
        ./BuildAll.sh
    fi
    # If dir contains a setup.py file, then build it.
    # setup.py will look for every .pyx extension and build it automatically.
    if test -f "setup.py"; then
        python3 setup.py build_ext --inplace
    fi
    cd ..
done

setup.py:

from setuptools import Extension, setup
from Cython.Build import cythonize
import glob

PYXFILES = glob.glob("*.pyx")
EXTNAMES = [i[:-4] for i in PYXFILES]

ext_list = []
for i in range(len(PYXFILES)):
    ext_list.append(
        Extension(
            EXTNAMES[i],
            [PYXFILES[i]],
            extra_compile_args=["-O3"]
        )
    )

setup(
    ext_modules = cythonize(
        ext_list,
        language_level = 3,
        build_dir = 'build',
        annotate = True
    )
)

classAC.cpp:

#include "classAC.hpp"

classAC::classAC()
    : data_(0)
{}

int classAC::data()
{
    return data_;
}

classAC.hpp:

#ifndef CLASSAC_H
#define CLASSAC_H

class classAC
{
private:
    int data_;
public:
    // Constructors
    // Null construct
    classAC();

    // Destructor
    ~classAC() = default;

    int data();
};

#endif // CLASSAC_H

classAC.pxd:

cdef extern from "classAC.cpp":
    pass

cdef extern from "classAC.hpp":
    cdef cppclass classAC:
        classAC()
        int data()

classA.pyx:

# distutils: language = c++

from classAC cimport *

cdef class ClassA:

    cdef classAC COBJ
    def __cinit__(self):
        pass

    def __init__(self):
        self.COBJ = classAC()

    def getAdata(self):
        return self.COBJ.data()

classBC.cpp:

#include "classBC.hpp"

classBC::classBC()
    : aobj_(classAC())
{}

classBC.hpp:

#ifndef CLASSBC_H
#define CLASSBC_H

#include "classAC.hpp"

class classBC
{
    /* Base class for an fvMesh */
private:
    classAC aobj_;
public:
    // Constructors
    classBC();

    // Destructor
    ~classBC() = default;
};

#endif // CLASSBC_H

classBC.pxd:

cdef extern from "classBC.cpp":
    pass

cdef extern from "classBC.hpp":
    cdef cppclass classBC:
        classBC()

classB.pyx:

# distutils: language = c++
# distutils: include_dirs = ../A

from classBC cimport *

cdef class ClassB:
    """
    """
    cdef classBC COBJ
    def __cinit__(self):
        pass

    def __init__(self):
        self.COBJ = classBC()

And if I try to run my main.py:

#!/usr/bin/python3

from src.A.classA import ClassA
from src.B.classB import ClassB

a = ClassA()
print(a.getAdata())

b = ClassB()

I get the following error:

Traceback (most recent call last):
  File "./main.py", line 4, in <module>
    from src.B.classB import ClassB
ImportError: <pathToThisFolder>/src/B/classB.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN7classACC1Ev

I have tried to link the classA.longname.so file to classBC but it did not helped.
Is it even possible to achieve this functionality? And if yes, how?
I would like to keep everything separated instead of having one huge module.
My goal would be to keep every setup.py file as it is and only add specific extension options using #distutils in the pyx files.

Thank you for your help, and just tell me if this structure is really bad...

question from:https://stackoverflow.com/questions/65857730/cython-reuse-wrapped-c-class

Answer 1

You'd normally use the "sources" option in setup.py or as a # distutils: sources = filename.cpp, another_filename.cpp directive in your .pyx file. This is documented in https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html#configuring-the-c-build. You do this instead of including the cpp file verbatim into your pxd file!

Thus your classB.pyx includes the line

# distutils: sources = classBC.cpp classAC.cpp

and thus they will both be compiled and linked into the extension module.

The distutils directive approach is probably better than putting it in setup.py because the dependencies are documented where they are used.

---

An alternative approach (which might be better for you based on the comments) would be to link all your C++ code together in a dynamic library and then link each of your separate Cython modules with that library (with distutils: libraries = .... Again, you're only telling including the header files (not the .cpp files) in your Cython code.