windows - Using C++ DLLs with different compiler versions

Question

This question is related to "How to make consistent dll binaries across VS versions ?"

• We have applications and DLLs built with VC6 and a new application built with VC9. The VC9-app has to use DLLs compiled with VC6, most of which are written in C and one in C++.
• The C++ lib is problematic due to name decoration/mangling issues.
• Compiling everything with VC9 is currently not an option as there appear to be some side effects. Resolving these would be quite time consuming.
• I can modify the C++ library, however it must be compiled with VC6.
• The C++ lib is essentially an OO-wrapper for another C library. The VC9-app uses some static functions as well as some non-static.

While the static functions can be handled with something like

// Header file
class DLL_API Foo
{
    int init();
}

extern "C"
{
    int DLL_API Foo_init();
}

// Implementation file
int Foo_init()
{
    return Foo::init();
}

it's not that easy with the non-static methods.

As I understand it, Chris Becke's suggestion of using a COM-like interface won't help me because the interface member names will still be decorated and thus inaccessible from a binary created with a different compiler. Am I right there?

Would the only solution be to write a C-style DLL interface using handlers to the objects or am I missing something? In that case, I guess, I would probably have less effort with directly using the wrapped C-library.

Answer 1

The biggest problem to consider when using a DLL compiled with a different C++ compiler than the calling EXE is memory allocation and object lifetime.

I'm assuming that you can get past the name mangling (and calling convention), which isn't difficult if you use a compiler with compatible mangling (I think VC6 is broadly compatible with VS2008), or if you use extern "C".

Where you'll run into problems is when you allocate something using new (or malloc) from the DLL, and then you return this to the caller. The caller's delete (or free) will attempt to free the object from a different heap. This will go horribly wrong.

You can either do a COM-style IFoo::Release thing, or a MyDllFree() thing. Both of these, because they call back into the DLL, will use the correct implementation of delete (or free()), so they'll delete the correct object.

Or, you can make sure that you use LocalAlloc (for example), so that the EXE and the DLL are using the same heap.