The project of the moment is to get serial messaging bidirectional in some kind of sane, useful way… I haven’t touched pretty much anything in astrid or pippi land in more than a week. (For a nice reason! I got really hooked on a book my cousin lent me last week.)

The hard part about this (as usual) is trying to decide on the structure of the data. None of my ideas are fancy. Some are decided:

• The data going over the serial wire will be a fixed size, like astrid’s lpmsg_t structs. It makes reading and buffering data coming in easier to manage and reason about when all payloads are the same size. I’m tempted to use a header + payload approach like libpippi buffers, but it would mean a more complicated situation for reading and writing data: reading enough bytes for the header, then parsing the header section to see how many more to read for the payload.
• Params are identified by an arbitrary integer that only has contextual meaning, like a file descriptor. It’s up to the instrument to define these numbers. (More on that later!)
• Params are delivered to instruments:
  • From the astrid console by typing commands like u amp=0.5 freq=234.2 which send an lpmsg_t struct of the type LPMSG_UPDATE and the msg member of the struct has the unparsed command string
  • From microcontrollers over the serial tty: the lpserialmsg_t struct comes in as a fixed payload of bytes, and gets relayed to the instrument as an encoded update message (more on this in a moment)

There are lots of things to figure out from there:

• Right now instruments listen for LPMSG_UPDATE messages, and have a special update callback to handle them. This allows any kind of arbitrary logic to receive and update params: the callback gets the instrument and the copy of the message, and it can do what it likes with that. However when the update message arrives I think it needs to already be encoded as a payload of bytes so that the update message can be decoded easily: in other words I want the update callbacks to only have to deal with one possible type of message.
• I started out with the idea that I’d just do a kind of MIDI-style ID/value payload, and just support a float type for all messages… that works great for ranged values (like the value of a knob) and for triggers and booleans and etc (like a switch, button, etc) but… not sequences, not arrays, not patterns and that sort of thing. A float can be an index into any kind of complex behavior… but the situation I’d like to support is to record onset times on the microcontroller (think: tap in a rhythm) and then send them over serial. So I think it makes sense to try to at least leave room to support that, even if it’s not supported in the first implementation.
• I guess an enum for the datatype makes sense… then the lpserialmsg_t struct could be something like: int for the ID, int for the type, and… maybe a union for the value? but that means changing the microcontroller firmware every time a new type for the value is added. (Because of the union type.) Maybe that’s OK, but keeping the value field as just a buffer of bytes is probably more flexible… In that case, how large should the buffer be?? A pattern type could express a pattern as bits in the buffer…

After typing all that out, I’m starting to wonder again if the payload should be variable, and I just need to deal up front with it in the serial reader?

Let’s say I use 64 bits for the value: it can hold a double, a long int, and a 64 step pattern… but if I wanted to hold a 61 step pattern…?

Maybe the last (or first) byte of the payload for a pattern type can hold the length of the pattern, which still leaves enough room for 60 steps to be encoded.

What about just using a header?? Maybe this is actually simpler…

• It means always decoding up to the header bytes first, then reading the payload bytes before being able to decode a full message. My implementation just writes bytes into a buffer until it has MSGSIZE bytes to decode then resets… but handling two steps in that process wouldn’t be impossible I guess. I could maybe track the bytes read in two places: header bytes, and payload bytes. Then once there are enough header bytes, start reading payload bytes, but wait to reset both counters until the (variable) number of payload bytes have been read, and the message is totally parsed.
• Once that’s worked out tho payloads could be… whatever. Do they need to be whatever? The msg field of the lpmsg_t only holds MAXMSG bytes anyway. (PIPE_BUF - the struct and other members)

No answers. None of it seems apparent.