Whew – I guess I’m about a month into rewriting all of astrid’s IPC. It has been a while coming, but I’m pretty happy with the new interfaces.

astrid_shared_resource_t * astrid_session_aquire_shared_resource(astrid_session_t * session, char * name, int resource_type);
int astrid_session_destroy_shared_resource(astrid_session_t * session, char * name, int resource_type);
int astrid_session_register_shared_resource(astrid_session_t * session, char * name, void * value, int resource_type, size_t size);
int astrid_session_set_shared_resource(astrid_session_t * session, char * name, void * value, int resource_type, size_t size);
int astrid_session_get_shared_resource(astrid_session_t * session, char * name, void * value, int resource_type, size_t size);
int astrid_session_release_shared_resource(astrid_session_t * session, char * name, astrid_shared_resource_t * resource);

Those replace all the old ipc value / id / sampler interfaces that were getting pretty out of hand.

Instead these new routines all take a resource name, size and type and are backed by either LMDB storage (in a database shared across all astrid instruments) or POSIX shared memory for anything larger. For a moment I thought why not just do everything with LMDB!? Some (most? who knows) of its speed comes from storing data as cache-or-smaller-sized chunks in an optimized tree structure, and it doesn’t do very well at all with large objects getting fragmented across that tree.

The POSIX shared memory segments – and LMDB writes – are synchronized with sempahores. I have such trouble with semaphores, but I think I shook all the deadlocks out again. ;-)

I’m finally now onto the python side of this, which involves splitting the Instrument abstraction up into Instrument and Renderer. The new IPC session is stored in a astrid_session_t struct on the instrument, but can be reopened from anywhere. That’s the first real win for python: no more serialization of state just to set up the render pool. Instead, each comrade opens its own handle to the shared session, and sends renders via the same RENDER_COMPLETE messages.

This should also let me eventually… eliminate copies during the render pipeline entirely??? All renders get copied like… too many times: once to copy into an lpbuffer_t for serialization, then again when copying the serialized buffer into shared memory, then again when deserializing it…

Since the new IPC routines have proper support for shared buffers, that can come down to a single copy when rendering is complete… and when I finish porting pippi’s SoundBuffer abstraction to be lpbuffer_t-backed instead of numpy-backed (not a huge priority but this is motivating me) then renders will be… zero copies!?

That’s exciting – it’s not a huge deal, but starts to have an impact when doing lots of long renders.

Log January 2025

(oops I missed September!)

Log August 2024