❯The Paradoxes of AI-first and Everything-LLM

In this article I want to attempt to analyze a logical inconsistency I've noticed concerning AI Engineering and the design of AI-first systems. These narratives are of various nature, but they have a common theme: AI proposes a fundamentally different way of doing engineering, to the point that we can replace human intervention with MCP servers or Agents, transitioning to a new way of writing software made of high-level, AI-first building blocks. These technologies may look titanic but they are all based on a common logical flaw, an Achilles' heel of sorts, a serious problem hard to detect.

Essentially, we look at technooogy to simplify things, but in many cases we end up having to manage more complexity than before. If we want adaptability, we lose simplicity, if we want to simplify we end up with an inflexible and dull system. This is a logical problem, not a technological one, and it cannot be fixed with more technology

It's useful to introduce the following point of view which is foundational for the rest of this article. Systems can be broken down and analyzed from two perspectives, the logical and the technological. Logical aspects come first, since technology implements and presupposes the logical ones.

Logic vs Implementation

The main observation I want to make is based on the conservation of complexity: there are fundamental limits to how we can organize the logic of our design before we hit a limit. We can move complexity to other subsystems but we can't eliminate it, this is true irrespective of the technology we use. The problem and its constraints demand e certain set of logical steps and state transitions. As long as we talk about AI though, we're looking at our system from the perspective of implementation rather than logic, and, for the most part, we can only hope to bring small improvements over time, directly related to the performance increase of the latest software releases. The logical layer is where major improvements can be made.

If we accept this point of view, certain conclusions become immediate.

From a technological perspective AI is surely a novelty, from the logical one, hardly so. I would challenge the idea that a tool, even AI, represents a novel logic that fundamentally changes how we think about engineering. In this sense, AI does little to "solve" design challenges.

The point is not being anti AI, but it's incorrect to point at the tool when the limit is the logic. This is a categorical mistake.

There is a small selection of tasks where AI unlocks new possibilities. Its value comes when the problem itself is fuzzy, ambiguous, or expensive to encode with rules. But even in those cases, AI is just one unit of a complex machine. As soon as we venture outside the implementation of the individual building block, we're again faced with the same issues I've pointed at above, we're still bound to logical limits, we still need to understand state transitions and failure modes and design accordingly. If nothing else AI exacerbates these problems. As of today, AI hasn't really impacted the logic of design, not yet, and the limitation is not technological and cannot easily be eliminated.

Conservation of Complexity

It's tempting to replace every component of our system with an LLM call via an MCP server or an AI equivalent, but that turns a local microsecond operation into a blocking network request. You are burning through compute and tokens, increasing security concerns, multiplying your API costs and exposing yourself to novel failure modes.

Prototyping with AI can feel like a breeze, no logic to program into your software, just let AI make the decision. That approach may work in development, but production systems are a very different environment. Once you have to meet reliability and performance guarantees, constraints become more apparent and the trade-offs begin to accumulate. At that point the idea that AI isn't bringing those magical advantages starts to seep in.

It's paradoxical that to resolve engineering problems we compensate with more complexity. Now that we have a central AI component, we must fine-tune it, generate datasets of features and versioned training datasets. Then we must evaluate the model. If the performance isn't satisfactory we must generate a dataset of rules and update the "long term memory" to continuously improve the model. All of this must be maintained of course.

AI first systems require an array of tools to be production ready:

→ Feature datasets
→ Test/Training datasets
→ Synthetic data generation
→ Fine-tuning
→ Prompt revisions and versioning
→ Datasets with Improvement rules
→ Monitoring output drift
→ Monitoring input dataset drift
→ Compensation logic for output randomness

Claiming you have an 'AI-first' system is appealing, but dialing back the rhetoric reveals a massive foundation of traditional software. Most of the heavy lifting is actually standard code and API calls designed to tame LLM randomness, measure performance, and keep the system running.

Conclusion

AI was supposed to simplify things. I'm arguing that accounting for errors, supervision, verification and model improvement, it often doesn't make things better, and that's not for superficial reasons that can be easily dismissed. The AI fever is leading us to paradoxes and architectural monstrosities. We may be persuaded by the fact that the initial design steps are simplified by replacing the costs of implementing logic with a fuzzy LLM system, but the conservation of complexity is soon going to bite back, forcing you to resort to compensatory systems that often add more burdens rather than solving problems.