The report that changed how a business made a decision

I was in a meeting where a multi-million dollar decision was almost finalized based on a shared story. The consensus was clear: our most valuable customers were churning over price. The anecdotes from the sales team were consistent and compelling. A major pricing overhaul seemed like the only logical, if painful, path forward.

These situations are a perfect demonstration of what the psychologist Daniel Kahneman identified as confirmation bias. When a story is repeated enough, we instinctively look for evidence that supports it. But a well-built data system has no bias. Its job is to reveal patterns, whether they fit the narrative or not.

The Architecture of a Question

Before building a report, my team had to re-architect the question. Leadership was asking, “How do we stop customers from leaving over price?” We needed to ask, “What are the actual, observable behaviors of customers who churn versus those who stay?” The difference is everything.

Answering our version required a clean, deterministic pipeline that stitched together three historically siloed sources: the billing system for churn events, product telemetry for user behavior, and support tickets for the customer's voice. This is the unglamorous work behind every meaningful insight. It's not about finding a magic model; it's about building a reliable factory for assembling facts.

Of course, it wasn't a clean process. Our first attempt to join telemetry with billing records failed quietly. A subtle change in customer ID formatting a year prior had corrupted the link for nearly 20% of the cohort, forcing us to build a messy, manual reconciliation process before we could even begin the analysis. That failure was a reminder that trust in data is built on wrestling with these boring, painful details.

The Chart That Changed the Plan

The final analysis boiled down to a single cohort chart. It segmented high-value customers who had churned and plotted their product usage against support ticket volume in the months leading up to their cancellation.

The pattern that emerged was undeniable. Price wasn't the root cause; a product failure was. A significant majority of these departing customers had all followed the same path: they adopted a specific new feature, their rate of application errors would spike, and they would file multiple support tickets about the same broken workflow. A month or two later, they would cancel. The pricing complaints were a symptom—an easy excuse for a frustrated user to give on the way out. The real disease was a high-friction experience in a feature we were actively marketing.

When we presented the chart, the energy in the room shifted. The data didn't have an opinion or an agenda. It was a direct, traceable line from a product flaw to lost revenue. The expensive pricing initiative was paused. A new, much cheaper, and more impactful plan was made: fix the broken feature.

Deterministic Systems Feed Better Decisions

The current hype cycle is focused on agentic AI systems that act on our behalf. But their value is entirely dependent on the quality of the deterministic systems that feed them. In this story, the "agent" was the human leadership team. Our job wasn't to build an AI to make the decision for them. It was to build a deterministic pipeline that produced a single, trustworthy piece of evidence so they could make a better one.

The report was a piece of automation. Its logic was encoded, version-controlled, and scheduled to run daily. Its boring reliability is what made it powerful. It created a trusted foundation for a creative, high-stakes human decision. This is the symbiotic relationship: deterministic systems produce truth, and agents—human or AI—act on that truth.

What Holds Up in Production

That experience remains a touchstone for me. The trendy part of our field is probabilistic models, but the durable value often comes from simple, deterministic systems of record. The report we built was, in essence, a data product. As Zhamak Dehghani outlines in her work on Data Mesh, treating data as a product means it must be trustworthy, understandable, and accessible to its consumer. Our report met that bar, which is why it had an impact.

The architecture that delivers clarity is what endures. A pipeline that joins customer records correctly every time is more valuable than a predictive model that’s 85% accurate but that no one can verify. The “boring” work of data modeling, clear lineage, and enforcing data contracts, as described in patterns like Data Mesh, is the infrastructure of credibility. It’s what survives the next platform shift. The goal isn’t always to automate the final decision. It’s to automate the delivery of irrefutable facts, so the people in the room can make the right one.

What to Take Away

Looking back, the lessons are repeatable and have little to do with specific technologies.

• Start with the question, not the data. Before writing code, rigorously challenge the initial business question. Building a perfect system to answer the wrong question is a high-cost form of waste.
• Engineer trust through transparency. The report landed because we could show every step of the data's journey. This transparency is a non-negotiable feature of any system meant to support major decisions.
• Build arguments, not just dashboards. Structure your data output to make a clear point. The goal is to close the loop from insight to action, and that requires a point of view grounded in auditable facts.
• Value deterministic clarity over probabilistic magic. An LLM might have found a correlation, but the hard-engineered link between telemetry, tickets, and billing was what made the finding undeniable. Build the boring, reliable foundation first.