jcardena.com › Blog › LLM Proposes, Code Enforces
184 posts
EN ES

LLM Proposes, Code Enforces

AI

Reliable agentic AI architecture: the model proposes, deterministic code enforces, RAG is retrieval done well, route by task. Trust lives in the code around the model.

LLM Proposes, Code Enforces

The most important design decision in any system I build with a language model is this: the model is the least trustworthy component, and I design it that way on purpose. That sounds harsh about a technology I use every day and genuinely love. It is not an insult. It is the assumption that lets me build things with a model that I am actually willing to leave running.

TL;DR: You do not earn trust by hoping the model behaves. You earn it with the code around the model. The model proposes an action, and deterministic code validates it and is the only thing allowed to actually do anything that matters. Add to that a clear eyed view of retrieval and a habit of routing work to the right model, and agentic systems become systems you can actually operate.

User IntentLLMProposesCode Validates& EnforcesAction Taken
The model never takes the real action. The code does, after checking.

The model proposes, the code enforces

Here is the pattern that does most of the work. The model never directly takes an action that is hard to undo. It proposes one. It says, in a structured way, "I think we should do this." Then ordinary deterministic code looks at that proposal, checks it against the rules, the permissions, the limits, the things that must always be true, and only then carries it out, or refuses.

This sounds simple and it makes all the difference. The model can be creative, occasionally wrong, even adversarially manipulated, and the worst it can do is propose something bad, which the code then declines. The intelligence lives in the model. The trust lives in the gate. I never ask "is the model reliable enough to be trusted with this," because I never trust it with the irreversible step at all. I trust the code that checks its homework, and code is something I can read, test, and reason about. In my own systems that rule is concrete: a model's output stays an untrusted proposal until typed code, a policy check, an eval, and an audit log have all accepted it.

Retrieval is just retrieval, and that is good news

A lot of what gets treated as mysterious in these systems is retrieval, the part where you find the right information to put in front of the model. People treat it as something genuinely new. It is not. It is information retrieval, a field that has been studied for decades, wearing newer clothes. The moment I started treating it that way, most of my problems got easier.

When retrieval feels like it is failing, it is almost never because the idea is broken. It is because the retrieval is bad. The chunks are the wrong size, so the right answer is split across two of them. The thing that decides relevance is matching on the wrong signal. There is no evaluation, so nobody actually knows whether the right document is being found. Pulling those apart and measuring them, treating retrieval quality as a number you track and improve rather than a vibe, is the difference between a system that grounds its answers and one that confidently makes things up. Storing vectors is not the work. Retrieving the right thing, and proving you did, is the work.

Streams of gold light retrieving and converging fragments into sharp focus
Retrieval is information retrieval: find the right thing, and prove you did.

Route the work to the right model

It is tempting to pick one strong model and send everything to it. It is also wasteful and slow. Most of the requests a real system handles are easy, and a smaller, cheaper, faster model handles them perfectly well. A smaller share are genuinely hard and deserve the expensive model. So I route by the task instead of by habit.

The factors are practical. Cost, because the easy majority should not be paying premium prices. Latency, because some answers need to be instant and some can take their time. And where the data is allowed to live, because some information cannot leave certain boundaries no matter how good the far away model is. Routing on those three turns a single expensive pipe into something that is cheaper, faster, and easier to keep compliant, with no loss on the requests that actually needed the big model.

Precise gold filtering gates and routing channels directing flowing rivers of light into ordered paths
Route by task and enforce with code: deterministic gates direct what the model is allowed to do.

Agentic without the chaos

An agent is just a loop. The model looks at the situation, picks a tool, the tool runs, the result comes back, and it goes again until the job is done. The power is obvious. So is the failure mode, which is a loop that never meaningfully ends, repeats itself, or wanders off doing expensive nonsense.

Keeping an agent useful is mostly about bounding it. A clear definition of done. A ceiling on how many steps it gets. Tools that are themselves guarded, so even inside the loop the propose and enforce rule still holds. And enough observability that I can see exactly what it did and why, after the fact. An agent you cannot observe is not an autonomous system, it is an unobservable one, and those two words mean very different things when something goes wrong at two in the morning.

Security is a trust boundary, not a feature

A core risk in these systems is that untrusted input meets a model that can trigger real actions. Someone hides an instruction inside a document or a web page, the model reads it as if it were a legitimate command, and now your helpful assistant is being quietly steered by a stranger. This is the same class of problem as every injection attack before it, with a twist: the model genuinely cannot reliably tell the difference between content and commands.

Which is exactly why the propose and enforce pattern matters so much here. If the model cannot take the dangerous action on its own, a hijacked model is far less dangerous, because the deterministic gate still applies the rules regardless of what the model was talked into proposing. I treat every place where outside text reaches the model as a boundary, I assume that text may be hostile, and I make sure the model's reach is limited by code that the outside text cannot touch.

The one that taught me this

I learned the value of the gate by not having one. An early agent of mine had a tool it could call freely, and one bad retrieval led it to take an action on the wrong target, repeatedly, very fast, because nothing was checking its proposals. Nothing catastrophic, but enough to make the lesson permanent. The fix was not a smarter model. It was a small piece of boring code between the model and the tool that asked, every time, "is this allowed, on this target, right now." That gate has saved me more times than any prompt I have ever written.

INPUTUser IntentUntrustedContentRETRIEVAL (RAG AS IR)ChunkEmbed &RetrieveGroundREASONINGModel RouterLLM ProposesENFORCEMENTValidatePermissionsLimitsACTIONGoverned Tools
The model proposes inside the reasoning layer. Only the enforcement layer can act.

The takeaway

The intelligence in these systems comes from the model. The trust comes from everything you build around it. Let the model propose and let code enforce. Treat retrieval as the well understood discipline it is and measure it. Route work to the model that fits it. Bound your agents and watch them. And remember that the model cannot tell a command from a paragraph, so the boundary has to be yours, in code, where you can prove it holds. Do that, and you can build things with a model that you are genuinely willing to leave running. That, to me, is the whole job.

JC
Juan Cardena
Enterprise AI & Agentic Systems Architect

Enterprise architect with 25 years across web, software, data, and AI. MIT CDAO ’25. Writing on agentic AI in production.