windows - Resolve managed and native stack trace - which API to use?

Question

This is continuation to my previous question - phase 2 so to say.

First question was here: Fast capture stack trace on windows / 64-bit / mixed mode

Now I have resolved a huge amount of stack traces and now wondering how to resolve symbol information of managed stack frames.

For native C++ side it's relatively simple -

First you specify which process from where to take symbols:

HANDLE g_hProcess = GetCurrentProcess();

Where you can replace process in run-time using code snipet like this:

g_hProcess = OpenProcess(PROCESS_ALL_ACCESS, FALSE, g_processId);

b = (g_hProcess != NULL );

if( !b )
    errInfo.AppendFormat(_T("Process id '%08X' is not running anymore."), g_processId );
else
    InitSymbolLoad();

And initialize symbol loading:

void InitSymbolLoad()
{
    SymInitialize(g_hProcess, NULL, TRUE);
    DWORD dwFlags = SymGetOptions();
    SymSetOptions(SymGetOptions() | SYMOPT_DEFERRED_LOADS | SYMOPT_NO_IMAGE_SEARCH);
}

And after that resolve native symbol , somehow like this:

extern HANDLE g_hProcess;

void StackFrame::Resolve()
{
    struct {
        union
        {
            SYMBOL_INFO symbol;
            char buf[sizeof(SYMBOL_INFO) + 1024];
        }u;
    }ImageSymbol = { 0 };

    HANDLE hProcess = g_hProcess;
    DWORD64 offsetFromSymbol = 0;

    ImageSymbol.u.symbol.SizeOfStruct = sizeof(SYMBOL_INFO);
    ImageSymbol.u.symbol.Name[0] = 0;
    ImageSymbol.u.symbol.MaxNameLen = sizeof(ImageSymbol) - sizeof(SYMBOL_INFO);
    SYMBOL_INFO* pSymInfo = &ImageSymbol.u.symbol;

    // Get file / line of source code.
    IMAGEHLP_LINE64 lineStr = { 0 };
    lineStr.SizeOfStruct = sizeof(IMAGEHLP_LINE64);

    function.clear();


    if( SymGetLineFromAddr64(hProcess, (DWORD64)ip, (DWORD*)&offsetFromSymbol, &lineStr) )
    {
        function = lineStr.FileName;
        function += "(";
        function += std::to_string((_ULonglong) lineStr.LineNumber).c_str();
        function += "): ";
    }

    // Successor of SymGetSymFromAddr64.
    if( SymFromAddr(hProcess, (DWORD64)ip, &offsetFromSymbol, pSymInfo) )
        function += ImageSymbol.u.symbol.Name;

}

This looks like working.

But now also managed stack frames.

There are two interfaces which I've located:

1. IDebugClient / GetNameByOffset

Mentioned in:

• http://www.codeproject.com/Articles/371137/A-Mixed-Mode-Stackwalk-with-the-IDebugClient-Inter (*) (Includes sample code)
• http://blog.steveniemitz.com/building-a-mixed-mode-stack-walker-part-1/

Used by:

• https://github.com/okigan/CrashInsight (Code not touched for 4 years)

• Mixed mode stackwalk article provides good example.

  2. IXCLRDATAProcess / GetRuntimeNameByAddress

• Mentioned also in two links above.

• Used by process hacker (GPL license, C style)

Implementation seems to reside in here:

• https://github.com/dotnet/coreclr/blob/master/src/debug/daccess/daccess.cpp (Based on commits this code is quite alive)

  3. ICorProfiler / ???

Mentioned at the end of (*) article.

Approach 1 seems to be quite old fashioned, also article (*) mentions some problems around it.

Approach 3 will probably require in-depth analysis of profiling API's. There is also one mention I have found about these API's - in here:

https://naughter.wordpress.com/2015/05/24/changes-in-the-windows-10-sdk-compared-to-windows-8-1-part-two/

· cor.h, cordebug.h/idl, CorError.h, CorHdr.h, corhlpr.h, corprof.h/idl, corpub.h/idl & corsym.h/idl: All of these header files have been removed. They are all the native mode COM interface to .NET.

This sentence I don't fully understand. Are those interfaces dead or replaced or what happened to them ?

So I guess based on my brief analysis approach 2 is only good / alive API interface which is worth of using ? Have you came across any problems related to those api's.

Answer 1

After walking through huge amount of code samples and interfaces, I've understood that there aren't any simple to use API interface. Code and API's developed for native C++ works only with native C++, and code and API's developed for managed code works only with managed code.

There is additionally problem of resolving stack trace afterwards might not work. You see - developer can generate code dynamically on fly using Jit engine / IL Generator, and dispose it as well - so after you have "void*" / instruction address - you should resolve symbolic information right away, not afterwards. But I'll leave this for time being, will assume that developer is not too fancy coder and not generating and disposing new code all the times, and FreeLibrary will not be called without need. (May be I can address this later on if I'll hook FreeLibrary / Jit components.)

Resolving function name was quite trivial, through IXCLRDataProcess with little bit of magic and luck - I was able to get function names, however - I want to expand it deeper - into exact source code path and source code line where code were executing, and this turned to be quite complex functionality to reach.

Finally I've hit upon source code where such thing were performed - and it was done here:

https://github.com/dotnet/coreclr/blob/master/src/ToolBox/SOS/Strike/util.cpp

GetLineByOffset is function name in that file.

I've analyzed, retuned and made my own solution from that source code, which I'm now attaching here now:

Updated code can be found from here: https://sourceforge.net/projects/diagnostic/

But here is just a snapshot of same code taken at some point of time:

ResolveStackM.h:

#pragma once
#include <afx.h>
#pragma warning (disable: 4091)     //dbghelp.h(1544): warning C4091: 'typedef ': ignored on left of '' when no variable is declared
#include <cor.h>                    //xclrdata.h requires this
#include "xclrdata.h"               //IXCLRDataProcess
#include <atlbase.h>                //CComPtr
#include <afxstr.h>                 //CString
#include <crosscomp.h>              //TCONTEXT
#include <Dbgeng.h>                 //IDebugClient
#pragma warning (default: 4091)

class ResoveStackM
{
public:
    ResoveStackM();
    ~ResoveStackM();
    void Close(void);

    bool InitSymbolResolver(HANDLE hProcess, CString& lastError);
    bool GetMethodName(void* ip, CStringA& methodName);
    bool GetManagedFileLineInfo(void* ip, CStringA& lineInfo);

    HMODULE mscordacwks_dll;
    CComPtr<IXCLRDataProcess> clrDataProcess;
    CComPtr<ICLRDataTarget> target;

    CComPtr<IDebugClient>       debugClient;
    CComQIPtr<IDebugControl>    debugControl;
    CComQIPtr<IDebugSymbols>    debugSymbols;
    CComQIPtr<IDebugSymbols3>   debugSymbols3;
};

//
// Typically applications don't need more than one instance of this. If you do, use your own copies.
//
extern ResoveStackM g_managedStackResolver;

ResolveStackM.cpp:

#include "ResolveStackM.h"
#include <Psapi.h>                      //EnumProcessModules
#include <string>                       //to_string
#pragma comment( lib, "dbgeng.lib" )


class CLRDataTarget : public ICLRDataTarget
{
public:
    ULONG refCount;
    bool bIsWow64;
    HANDLE hProcess;

    CLRDataTarget( HANDLE _hProcess, bool _bIsWow64 ) :
        refCount(1), 
        bIsWow64(_bIsWow64),
        hProcess(_hProcess)
    {
    }

    HRESULT STDMETHODCALLTYPE QueryInterface( REFIID riid, PVOID* ppvObject)
    {
        if ( IsEqualIID(riid, IID_IUnknown) || IsEqualIID(riid, __uuidof(ICLRDataTarget)) )
        {
            AddRef();
            *ppvObject = this;
            return S_OK;
        }

        *ppvObject = NULL;
        return E_NOINTERFACE;
    }

    ULONG STDMETHODCALLTYPE AddRef( void)
    {
        return ++refCount;
    }

    ULONG STDMETHODCALLTYPE Release( void)
    {
        refCount--;

        if( refCount == 0 )
            delete this;

        return refCount;
    }

    virtual HRESULT STDMETHODCALLTYPE GetMachineType( ULONG32 *machineType )
    {
        #ifdef _WIN64
            if (!bIsWow64)
                *machineType = IMAGE_FILE_MACHINE_AMD64;
            else
                *machineType = IMAGE_FILE_MACHINE_I386;
        #else
            *machineType = IMAGE_FILE_MACHINE_I386;
        #endif

        return S_OK;
    }

    virtual HRESULT STDMETHODCALLTYPE GetPointerSize( ULONG32* pointerSize )
    {
#ifdef _WIN64
    if (!bIsWow64)
#endif
        *pointerSize = sizeof(PVOID);
#ifdef _WIN64
    else
        *pointerSize = sizeof(ULONG);
#endif
        return S_OK;
    }

    virtual HRESULT STDMETHODCALLTYPE GetImageBase( LPCWSTR imagePath, CLRDATA_ADDRESS *baseAddress )
    {
        HMODULE dlls[1024] = { 0 };
        DWORD nItems = 0;
        wchar_t path[ MAX_PATH ];
        DWORD whatToList = LIST_MODULES_ALL;

        if( bIsWow64 )
            whatToList = LIST_MODULES_32BIT;

        if( !EnumProcessModulesEx( hProcess, dlls, sizeof(dlls), &nItems, whatToList ) )
        {
            DWORD err = GetLastError();
            return HRESULT_FROM_WIN32(err);
        }

        nItems /= sizeof(HMODULE);
        for( unsigned int i = 0; i < nItems; i++ )
        {
            path[0] = 0;
            if( GetModuleFileNameEx(hProcess, dlls[i], path, sizeof(path) / sizeof(path[0])) )
            {
                wchar_t* pDll = wcsrchr( path, L'\');
                if (pDll) pDll++;

                if (_wcsicmp(imagePath, path) == 0 || _wcsicmp(imagePath, pDll) == 0)
                {
                    *baseAddress = (CLRDATA_ADDRESS) dlls[i];
                    return S_OK;
                }
            }
        }
        return E_FAIL;
    }

    virtual HRESULT STDMETHODCALLTYPE ReadVirtual( CLRDATA_ADDRESS address, BYTE *buffer, ULONG32 bytesRequested, ULONG32 *bytesRead )
    {
        SIZE_T readed;

        if( !ReadProcessMemory(hProcess, (void*)address, buffer, bytesRequested, &readed) )
            return HRESULT_FROM_WIN32( GetLastError() );

        *bytesRead = (ULONG32) readed;
        return S_OK;
    }

    virtual HRESULT STDMETHODCALLTYPE WriteVirtual( CLRDATA_ADDRESS address, BYTE *buffer, ULONG32 bytesRequested, ULONG32 *bytesWritten )
    {
        return E_NOTIMPL;
    }

    virtual HRESULT STDMETHODCALLTYPE GetTLSValue( ULONG32 threadID, ULONG32 index, CLRDATA_ADDRESS *value )
    {
        return E_NOTIMPL;
    }

    virtual HRESULT STDMETHODCALLTYPE SetTLSValue( ULONG32 threadID, ULONG32 index, CLRDATA_ADDRESS value )
    {
        return E_NOTIMPL;
    }

    virtual HRESULT STDMETHODCALLTYPE GetCurrentThreadID( ULONG32 *threadID )
    {
        return E_NOTIMPL;
    }

    virtual HRESULT STDMETHODCALLTYPE GetThreadContext( ULONG32 threadID, ULONG32 contextFlags, ULONG32 contextSize, BYTE *context )
    {
        return E_NOTIMPL;
    }

    virtual HRESULT STDMETHODCALLTYPE SetThreadContext( ULONG32 threadID, ULONG32 contextSize, BYTE *context)
    {
        return E_NOTIMPL;
    }

    virtual HRESULT STDMETHODCALLTYPE Request( ULONG32 reqCode, ULONG32 inBufferSize, BYTE *inBuffer, ULONG32 outBufferSize, BYTE *outBuffer)
    {
        return E_NOTIMPL;
    }
}; //CLRDataTarget





ResoveStackM::ResoveStackM() :
    mscordacwks_dll(0)
{

}

ResoveStackM::~ResoveStackM()
{
    Close();
}

void ResoveStackM::Close( void )
{
    clrDataProcess.Release();
    target.Release();
    debugClient.Release();

    if( mscordacwks_dll != 0 )
    {
        FreeLibrary(mscordacwks_dll);
        mscordacwks_dll = 0;
    }
}

bool ResoveStackM::InitSymbolResolver(HANDLE hProcess, CString& lastError)
{
    wchar_t path[ MAX_PATH ] = { 0 };

    // According to process hacker - mscoree.dll must be loaded before loading mscordacwks.dll.
    // It's enough if base application is managed.

    if( GetWindowsDirectoryW(path, sizeof(path)/sizeof(wchar_t) ) == 0 )
        return false;   //Unlikely to fail.

#ifdef _WIN64
    wcscat(path, L"\Microsoft.NET\Framework64\v4.0.30319\mscordacwks.dll");
#else
    wcscat(path, L"\Microsoft.NET\Framework\v4.0.30319\mscordacwks.dll");
#endif

    mscordacwks_dll = LoadLibraryW(path);
    PFN_CLRDataCreateInstance pCLRCreateInstance = 0;

    if( mscordacwks_dll != 0 )
        pCLRCreateInstance = (PFN_CLRDataCreateInstance) GetProcAddress(mscordacwks_dll, "CLRDataCreateInstance");

    if( mscordacwks_dll == 0 || pCLRCreateInstance == 0)
    {
        lastError.Format(L"Required dll mscordacwks.dll from .NET4 installation was not found (%s)", path);
        Close();
        return false;
    }

    BOOL isWow64 = FALSE;
    IsWow64Process(hProcess, &isWow64);
    target.Attach( new CLRDataTarget(hProcess, isWow64 != FALSE) );

    HRESULT hr = pCLRCreateInstance(__uuidof(IXCLRDataProcess), target, (void**)&clrDataProcess );

    if( FAILED(hr) )
    {
        lastError.Format(L"Failed to initialize mscordacwks.dll for symbol resolving (%08X)", hr);
        Close();
        return false;
    }

    hr = DebugCreate(__uuidof(IDebugClient), (void**)&debugClient);
    if (FAILED(hr))
    {
        lastError.Format(_T("Could retrieve symbolic debug information using dbgeng.dll (Error code: 0x%08X)"), hr);
        return false;
    }

    DWORD processId = GetProcessId(hProcess);
    const ULONG64 LOCAL_SERVER = 0;
    int flags = DEBUG_ATTACH_NONINVASIVE | DEBUG_ATTACH_NONINVASIVE_NO_SUSPEND;

    hr = debugClient->AttachProcess(LOCAL_SERVER, processId, flags);
    if (hr != S_OK)
    {
        lastError.Format(_T("Could attach to process 0x%X (Error code: 0x%08X)"), processId, hr);
        Close();
        return false;
    }

    debugControl = debugClient;

    hr = debugControl->SetExecutionStatus(DEBUG_STATUS_GO);
    if ((hr = debugControl->WaitForEvent(DEBUG_WAIT_DEFAULT, INFINITE)) != S_OK)
    {
        return false;
    }

    debugSymbols3 = debugClient;
    debugSymbols  = debugClient;
    // if debugSymbols3 == NULL - GetManagedFileLineInfo will not work
    return true;
} //Init

struct ImageInfo
{
    ULONG64 modBase;
};

// Based on a native offset, passed in the first argument this function
// identifies the corresponding source file name and line number.
bool ResoveStackM::GetManagedFileLineInfo( void* ip, CStringA& lineInfo )
{
    ULONG lineN = 0;
    char path[MAX_PATH];
    ULONG64 dispacement = 0;

    CComPtr<IXCLRDataMethodInstance> method;
    if (!debugSymbols || !debugSymbols3)
        return false;

    // Get managed method by address
    CLRDATA_ENUM methEnum;
    HRESULT hr = clrDataProcess->StartEnumMethodInstancesByAddress((ULONG64)ip, NULL, &methEnum);
    if( hr == S_OK )
    {
        hr = clrDataProcess->EnumMethodInstanceByAddress(&methEnum, &method);
        clrDataProcess->EndEnumMethodInstancesByAddress(methEnum);
    }

    if (!method)
        goto lDefaultFallback;

    ULONG32 ilOffsets = 0;
    hr = method->GetILOffsetsByAddress((CLRDATA_ADDRESS)ip, 1, NULL, &ilOffsets);

    switch( (long)ilOffsets )
    {
        case CLRDATA_IL_OFFSET_NO_MAPPING:
            goto lDefaultFallback;

        case CLRDATA_IL_OFFSET_PROLOG:
            // Treat all of the prologue as part of the first source line.
            ilOffsets = 0;