I thought a little more about the luvsound backend tonight. Using plain text files as the canonical representation for each entity: eg releases, orders, etc.
Doing reads and writes can be done directly in the filesystem, or through a cache of each record in an sqlite database.
Writes to the database would be serialized into a write queue, with a single writer process that could be optionally paused during manual editing. Writes in the filesystem would be synchronized back to the database (using paths as keys) through a signal mechanism like inotify.
Tools built around the collection of files would use the database cache for search and filtering operations on the files.
The raw file format should be something simple and human-writeable. I sketched out a basic idea based loosely on HTTP headers, but I’m not sure if the header/body paradigm really makes sense. What might be simpler is to refer to adjacent files or directories of files in the main file, so images, HTML, markdown, audio etc can be included easily and pulled in when rendering the view for the given resource.
This sort of suggests some interchange format which can be piped to other tools to render different types of views sort of hypermedia style. I’m not sure what that would be? But it would go something like:
source file -> interchange format -> target format
So the core system would have source -> interchange translators which would do things like typing the resource, validation and etc, and renderers for target formats can be written to consume something standardized and more easily machine-readable.
The pipeline paradigm is intentionally meant to be UNIX-y so that some of these renderers could actually be existing tools…
Building a given client or server application around these core files then would be something like plugging together pipelines of these tools…
That all needs more thought! This is somewhere in line with where I’d like astrid programs to go eventually too. It would be very cool to support pipelines of streams for certain types of resources, handling them as much as a core part of the system as any other type of resource would make doing things like distributing procedural audio on demand etc just another optional part of the system…