The Model Is Not The Problem: The Engine Is The Problem

Thanks to Charles Sullivan for giving this talk! The Model Is Not The Problem: The Engine Is The Problem There is a massive disconnect in AI. We have five discrete but interconnected problems with inference in 2026: The streaming problem. Inference is a black box: wall of text in, pull the lever, wall of text out. No streaming, no back-and-forth, no iterative progress. The memory problem. The LLM can't remember anything. It pretends to remember by re-reading context on every request, but it doesn't actually remember. The learning problem. The LLM can't internalize anything. If the capital of France were to change tomorrow, we would have to add the new capital to the system prompt to be re-read on every pass. And learning new skills is completely out of the question. The text problem. Underpinning all of this is the fact that all "memories" (e.g. from RAG) and "skills" (e.g. tool calls) are plain text. That would be like remembering a concert by reading notes about it. That or playing a video game by typing, \pressButton('x')\ when you want Mario to jump. * The reputation problem. LLMs don't understand "cheating" because they cannot learn, and thus they can never internalize when humans are displeased with their actions. While LLMs are perfectly capable of empathy, they are not endowed with the biological imperative of reputation and thus continue to confound us with perfectly logical but morally-reprehensible actions (e.g. the OpenAI bot hack event). The industry keeps trying to solve these problems using more models. We have embedding models to try to solve the memory problem. They are a poor substitute for the first-hand experience behind the text. We use Jev to solve the press-button-to-jump problem, but of course Jev can't learn patterns, so every interaction is that of a beginner. We use thinking and tool calling turns to solve the streaming problem, but streaming the output in segments only soves one tiny corner of the problem--we still can't…