Someone had a good idea.

I’ve been waffling about adding another unwanted think piece to the LLM pile, but then I ranted at my coworker about it today and it came out as a pretty good summary of my experience with LLMs I guess! So here it is:

I actually have made serious attempts to “vibe code” as experiments – those experiments are over though. I tried a few small side projects over the last year – scripts to manage my personal finances (I almost couldn’t pay rent the first month because I trusted the LLM too much and mismanaged my bills) and extensions to my personal reminder / note-taking system (after several frustrating and nearly useless months it’s all fully disabled now) and I even tried to use it to prototype an esp32 project, that one had some interesting aspects – there is a lot of boilerplate in embedded programming, so it was somewhat useful just as a “hello world” generator for various components that I could study… but ultimately it made a total mess of the code and I’m starting over completely on the project… the experiments are over, but I did discover legitimate useful cases in the process. if i had to summarize it would be: use it for problems rooted in pattern-matching, always give it obvious and concrete examples (so start by writing code manually always, then let the LLM fill out the boring parts, or show it code you’ve already written and explain how it should differ) and never let a session last longer than a few back-and-forths… it’s actually quite useful for a lot of things in the end, even though I occasionally fall into a trap of thinking I can do more and then waste time cleaning up the inevitable mess. on a short leash so to speak it’s pretty useful… but I wouldn’t actually bother if speed / time wasn’t important for . i don’t plan to use it in any of my personal work, i don’t see the point. it even seems actively harmful.

Oh! the other really important one is you have to ground everything in concrete tool calls and scripts or automated tests that the LLM can use to self-evaluate. Otherwise it just makes random shit up like always. The models themselves are incrementally better, but they fundamentally still have all the same problems. The tool harnesses are what really changed in the last year – the whole “agent” paradigm is basically based around the idea of using little islands of context by calling out to other LLMs and getting useful grounding context (the results of running a series of exploratory tool calls, or a web search or etc) to move forward with. They’re completely useless without doing that.

That’s also the sort of thing that will make tiny local models actually practically useful (for certain tasks of course) – a couple weeks ago I tried training another small model. this time on synthetic documents that showed examples of natural language search queries that get translated into small bash scripts with typical find/grep/etc commands. the experiment this time was to combine that with a (vibe coded, lol, but the script is ~100 lines) bash script which just takes a path and a natural language query as inputs, and feeds them to the LLM which is trained to only produce executable bash outputs – so at that point I can just do static analysis, run it through shellcheck etc and call the LLM in a loop until it produces something valid / executable if needed – then the final part of the script is just: it shows me the bash script and I can say “ok run it” or reject it. that’s the core interaction of something like claude code when it comes down to it, plus tool calling (which is just parsing the output of the LLM and calling tools the old fashioned way – same flow as the bash script generation, but with special symbols trained into the LLM for tool calling.)

I think what was most interesting me through this whole process was discovering certain kinds of errors of accumulation that I hadn’t really encountered like this before. It’s like the experience of technical debt, but it happens on the time scale of days not months or years, and you also don’t get the benefit of deep interaction with the system, so you can come out from supervising every detail of the code, and have correct code on the local scale so to speak, but in the process create architectural problems that aren’t always easy to spot until it’s too late and you have a mess to clean up. what’s interesting is how easy it is to do, but also how it doesn’t really have to do with correctness per se – the code might fail in seriously weird ways and be inscrutable, but will generally be correct in each isolated scope you study, the problems emerge in composition at the large scale. which makes total sense because these things don’t think, and that’s a level beyond where rote pattern matching will yield anything but insanity.