Fumbling once again to take a python unicode string and pass it to a C function that wants char * this morning I figured I’d write it down for next time.

The basic dance is to encode the string as bytes, store a reference to this as a python object to prevent it from being garbage collected and then assign it to a cdefed char * pointer and the cython compiler takes the pointer from the bytes object and passes to the C function, keeping the reference around as long as the intermediary python object is alive. (So if the string needs to stay around, malloc a copy first!)

def something(str a_unicode_string_from_python):
    # keep a reference to the bytes object around to prevent garbage collection until the function returns
    bytes_object_reference = a_unicode_string_from_python.encode('UTF-8')

    # The cython compiler knows how to find the pointer to the internal string of the bytes object
    cdef char * a_char_pointer = bytes_object_reference

    # And now we have our char pointer
    a_c_function_that_wants_a_char_pointer(a_char_pointer)

    # After this point, the memory from the bytes object is freed so make a copy if the string 
    # needs to be long-lived beyond this point!
    return

Edit Mar 20th: If anyone besides me is reading this, beware that this just passes unicode bytes happily along. It works fine for ASCII inputs, but if you use code points above the ASCII range that need more than one byte, it’ll get passed along as garbage unless the bytes are being explicitly decoded as unicode on the C side.

If ASCII is really all that’s needed, doing a_unicode_string_from_python.decode('ascii', errors='replace').encode('utf-8') will first decode the unicode string as ASCII bytes and replace any multibyte characters with ? (or use ignore to strip them out 1.) then encode('utf-8') encodes them to bytes again before handing off to the cython compiler which gets the pointer to the underlying bytes and assigns that to a_char_pointer.