python - How to stop memory leaks when using `as_ptr()`?

Question

Since it's my first time learning systems programming, I'm having a hard time wrapping my head around the rules. Now, I got confused about memory leaks. Let's consider an example. Say, Rust is throwing a pointer (to a string) which Python is gonna catch.

In Rust, (I'm just sending the pointer of the CString)

use std::ffi::CString;

pub extern fn do_something() -> *const c_char {
    CString::new(some_string).unwrap().as_ptr()
}

In Python, (I'm dereferencing the pointer)

def call_rust():
    lib = ctypes.cdll.LoadLibrary(rustLib)
    lib.do_something.restype = ctypes.c_void_p
    c_pointer = lib.do_something()
    some_string = ctypes.c_char_p(c_pointer).value

Now, my question is about freeing the memory. I thought it should be freed in Python, but then ownership pops in. Because, as_ptr seems to take an immutable reference. So, I got confused about whether I should free the memory in Rust or Python (or both?). If it's gonna be Rust, then how should I go about freeing it when the control flow has landed back into Python?

Answer 1

Your Rust function do_something constructs a temporary CString, takes a pointer into it, and then drops the CString. The *const c_char is invalid from the instant you return it. If you're on nightly, you probably want CString#into_ptr instead of CString#as_ptr, as the former consumes the CString without deallocating the memory. On stable, you can mem::forget the CString. Then you can worry about who is supposed to free it.

Freeing from Python will be tricky or impossible, since Rust may use a different allocator. The best approach would be to expose a Rust function that takes a c_char pointer, constructs a CString for that pointer (rather than copying the data into a new allocation), and drops it. Unfortunately the middle part (creating the CString) seems impossible on stable for now: CString::from_ptr is unstable.

A workaround would to pass (a pointer to) the entire CString to Python and provide an accessor function to get the char pointer from it. You simply need to box the CString and transmute the box to a raw pointer. Then you can have another function that transmutes the pointer back to a box and lets it drop.