Getting closer to integrating the embedded python interpreter in the new style astrid C instruments. I’m working on removing redis which was used to send serialized buffers from the renderer processes to a single dac program for playback.

Instead of redis, I’m saving the serialized buffer as a shared memory segment, and sending a new RENDER_COMPLETE message on the instrument’s message queue with the address of the shared memory segment. The receiver is the instrument message handler which deserializes the buffer and places it into the scheduler for playback in the jack callback. Those buffer IDs could be stored or passed around elsewhere too though, like the LMDB session. I’m hoping this’ll make a decent backend to build a shared sampler interface and built-in input ringbuffers, etc.

Before that I had some embedded python quirks to work through. The new PyConfig structure makes it pretty easy to configure an embedded python interpreter that behaves like it’s the standard python interpreter in whatever environment you happen to be in.

The last bit is important for astrid, since pippi isn’t a trivial package to install (though once libpippi is more fully integrated that situation should improve) and re-building pippi and its dependencies (hello numpy!) along with the embedded interpeter is a much bigger pain in the butt. It’s nicer to keep them seperate for now, and PyConfig helps a whole lot with that.

Essentially the only crucial bit is to install pippi in a venv (or system-wide if you know what you’re doing) and then run the astrid C programs with the venv activated. PyConfig uses the venv environment and the embedded interpreter acts just like it’s the normal python installed in your system.

From the new linenoise repl: (^_- is the winky astrid command prompt – also notice the play params get ignored since they’re still only parsed in the cython layer at the moment)

^_- p foo=bar
msg: p
^_-

The pulsar application gets the play message from the REPL and calls the internal cython interface to begin a render.

Mar 15 08:24:39 lake pulsar[514199]: MSG: pulsar
Mar 15 08:24:39 lake pulsar[514199]: MSG: 6 (msg.voice_id)
Mar 15 08:24:39 lake pulsar[514199]: MSG: 1 (msg.type)
Mar 15 08:24:39 lake pulsar[514199]: MSG: play

The embedded interpreter executes this python instrument script:

from pippi import dsp, oscs

def play(ctx):
    ctx.log('Rendering simple tone')
    yield oscs.SineOsc(freq=330, amp=0.5).play(1).env('pluckout')

It logs a message from inside the python render process (the first message is from the intermediary cython module layer which collects and executes all the play methods in an instrument script, does live reloading and etc).

Mar 15 08:24:39 lake astrid-pulsar[514199]: rendering event <cyrenderer.Instrument object at 0x797f174dcd00> w/params b''
Mar 15 08:24:39 lake astrid-pulsar[514199]: ctx.log[simple] Rendering simple tone

In the cython layer, the buffer is serialized and passed to an astrid library routine which generates a buffer ID, saves the buffer string into shared memory and sends the RENDER_COMPLETE message on the pulsar message queue.

The pulsar message handler thread gets the message and logs that it arrived.

Mar 15 08:24:39 lake pulsar[514199]: MSG: pulsar
Mar 15 08:24:39 lake pulsar[514199]: MSG: 0 (msg.voice_id)
Mar 15 08:24:39 lake pulsar[514199]: MSG: 7 (msg.type)
Mar 15 08:24:39 lake pulsar[514199]: MSG: render complete
Mar 15 08:24:39 lake pulsar[514199]: pulsar-7-0-0-0-0-0-0-0

Next up: deserialize the buffer string and place it into the scheduler. Should be way easier than the other parts of this process but my morning astrid time is over for today. :-)

Update: I made those changes after work today and… seems to be going ok?