When data and AI must not fail, I reach for Rust

After two decades building systems where the data has to be right and the latency has to be boring, I've grown wary of stacks that are quick to demo and slow to trust. The last few years added a new wrinkle: we now wire probabilistic models and autonomous agents into the same pipelines that move money, records, and decisions. The interesting question stopped being “can the model do it” and became “what is this running on when it has to not fail at three in the morning.”

For the parts that must not fail, I increasingly reach for Rust. Not everywhere — most of my data and AI work is still Python, and that is correct. But underneath the probabilistic top there is a deterministic spine, and Rust is where I have been putting it.

The probabilistic top, the deterministic bottom

An agentic system is non-deterministic by design. You ask a model to reason, and you accept that the same input can produce different paths. That is the feature, not the bug. But the moment that reasoning touches a pipeline, it inherits a set of obligations that are not negotiable: the input must be validated, the schema must hold, the call must be idempotent, the rate budget must be respected, and a malformed record must never silently corrupt the next one.

Those obligations are deterministic. Writing them in a language that lets a null slip through, a buffer overrun go unnoticed, or a data race quietly scramble state is borrowing trouble you will repay during an incident. The probabilistic part is allowed to be uncertain. The plumbing is not.

Where Rust earns its place in a data stack

Three places have consistently paid for themselves. First, ingestion and parsing at volume — the unglamorous work of turning messy bytes into trustworthy records, where a single mishandled edge case becomes tomorrow's data-quality ticket. Second, hot-path transforms that run on every event. Third, long-running services that cannot afford the pauses a garbage collector introduces.

What Rust buys you in all three is memory safety without a garbage collector. Tail latency stays predictable, memory stays bounded, and the cost curve stays flat as traffic grows instead of bending upward the night you get noticed. You pay for that safety once, at compile time, instead of repeatedly, in production.

Rust at the edge of the agent

The most useful place I have found for Rust in AI work is the boundary around the agent: the deterministic tools and validators it calls. An agent proposes; a typed, total Rust function disposes. The model can suggest a query, a transform, a payment amount — and a small, fast, exhaustively-tested validator decides whether that suggestion is allowed to touch anything real.

This is also where the language wars dissolve. You do not have to abandon the Python ML ecosystem to get Rust where it matters. Tools like PyO3 let you keep the notebooks, the training, and the model serving in Python and call into Rust for the parts that must be fast and safe. The agent stays flexible; its hands stay clean.

What it costs, honestly

Rust is not free, and pretending otherwise is how teams end up resenting it. Compile times are real. The borrow checker is a genuine learning curve. The hiring pool is smaller. I do not write glue, experiments, or notebooks in it — that would be using a torque wrench to butter toast.

The rule I use is simple: if a component sits on a hot path, holds an invariant that must never break, or runs unattended for months, it is a Rust candidate. Everything else — the exploration, the orchestration, the parts that change weekly — stays in Python, where iteration speed is the right thing to optimize for.

The architecture I keep coming back to

Put together, the shape is consistent across projects: a probabilistic, fast-moving layer of models and agents on top, and a deterministic, memory-safe spine of ingestion, validation, and hot-path processing underneath — with a clean interop seam between them rather than a wall.

None of this is about a language being fashionable. It is about knowing which parts of a system are allowed to be uncertain and which are not, and choosing tools that match. The agents get to be clever. The substrate gets to be boring. That division of labor is what lets the whole thing survive contact with production.

If this way of thinking about durable data and AI architecture is useful to you, the rest of the essays live at blog.jcardena.com.