Enterprise AI: Moving from Experimentation to Production

Building an AI prototype has become considerably easier. A competent team can now put together a chatbot over a knowledge base, an agent that drafts a first-cut report, or a classifier that flags exceptions for review, in a matter of weeks. The demo works. Stakeholders are impressed. And in a large number of organisations, that is where it stays: a promising prototype that never quite becomes something the business actually runs on.

That gap is not primarily a model problem. MIT’s NANDA initiative studied enterprise generative AI deployments through 2025 and reported limited measurable profit-and-loss impact across most of the deployments it examined, pointing to integration, workflow fit and organisational learning, not model quality, as the more significant barriers (MIT NANDA, The GenAI Divide: State of AI in Business 2025). The model did the part it was asked to do. The system around it wasn’t built to be depended on.

This article is about that distinction: what actually changes, technically and organisationally, when an AI initiative moves from “we proved it can work” to “the business depends on it working correctly, every day, inside a real process.”

The question a demonstration cannot answer

A demonstration proves capability. Under conditions the team controls, with inputs chosen to be reasonably representative, the model produces a plausible output. That is real progress, and it is also a narrow claim.

A production system has to answer a different question: can the organisation depend on it? Depend on it to behave sensibly when the input is messy, the underlying data is incomplete, a user asks something the designers didn’t anticipate, or nobody from the original build team is in the room to explain an odd result. Dependency implies repeatability under real variation, and that is precisely what a demonstration is not built to test.

This isn’t an argument against prototyping. Fast, disposable prototypes are how you learn whether an idea has legs before committing engineering budget to it. The mistake is treating a successful prototype as a smaller, unfinished version of the production system. It’s usually a different kind of artefact, built to answer “can this work” rather than “will this keep working.” A technically impressive prototype can still be operationally unusable, not because it was built badly, but because it was built to answer the wrong question.

Start with the business problem, not the model

The organisations that get stuck tend to start with the technology: “what can we do with an LLM?” The ones that get somewhere start with a business problem that already has a shape: a defined process, a user, a decision or task currently done by a person, and a reasonably clear idea of what “better” would look like.

Before any architecture decision, it’s worth being explicit about a handful of things that are easy to leave implicit: what the business objective actually is, who the user or process owner is, what value is expected and how it will be measured, what an acceptable failure looks like, and what “good enough to ship” means for this specific task. That last one matters more than it sounds. A system drafting internal meeting notes can tolerate a different failure mode than one that classifies transactions for a compliance review. Treating both the same way, with one bar that’s either too strict or too loose, is how projects either stall in endless refinement or ship something the business can’t actually trust.

Data becomes an engineering problem

Once the business problem is clear, most of the early technical risk sits in the data, not the model. Enterprise data was rarely organised with an AI system as its intended consumer. Getting it into a usable state means working through data quality, access permissions, provenance, and how current the information actually is relative to how current it needs to be.

Permissions deserve particular attention because they’re easy to get wrong quietly. A retrieval system that’s technically accurate but occasionally surfaces information the requesting user wasn’t supposed to see is a governance failure, not a model failure, and it usually traces back to access control being bolted on after the fact rather than designed in from the start. Similarly, “the data is available” and “the data is fit for this purpose in this business context” are different claims: a field can be technically populated and still mean something subtly different than what the system assumes it means.

Architecture is more than choosing a model

Selecting a large language model is one architectural decision among several, and often not the first one that matters. A working enterprise AI system is more commonly a composition: deterministic software where the logic is genuinely deterministic, classical machine learning where the task is a good statistical fit, an LLM where language understanding or generation is actually required, retrieval-augmented generation where the answer needs to be grounded in specific organisational content, tools and APIs where the system needs to act rather than just respond, a workflow or orchestration layer to sequence all of that, and human review where the risk profile calls for it.

The architectural question worth asking early is not “how do we use an LLM here” but “which of these components does this specific problem actually need.” Not every problem needs an agent, and greater autonomy is not automatically better architecture. A well-scoped deterministic step, wired correctly into the process, is often more reliable and considerably cheaper to operate than a more autonomous system asked to make the same decision without one. Gartner made a related prediction in 2025: that over 40% of agentic AI projects would be cancelled by the end of 2027, citing escalating cost, unclear business value and inadequate risk controls as the drivers, not a lack of model capability (Gartner, June 2025). Deterministic software remains valuable precisely because it behaves the same way every time; that property doesn’t stop being useful just because more capable components are now available. We’ve written more about the range of components we typically combine for enterprise clients on what we do.

Grounding: retrieving isn’t the same as knowing

For any application that has to answer questions about an organisation’s own information (its policies, its processes, its historical decisions), the generated output usually needs to be grounded in specific, authorised organisational content, not the model’s general training.

It’s worth being precise about what grounding actually does. It doesn’t make the model “know” the organisation the way an experienced employee does. It constrains generation to draw from a defined, retrievable set of authorised material, and it makes it possible to trace an answer back to a source. Those are different properties, and conflating them leads to overconfidence: a well-grounded system can still retrieve the wrong passage, retrieve an outdated one, or retrieve something technically relevant but insufficient to answer the question correctly. Retrieval quality and answer quality are related but not identical. A system can retrieve the right document and still generate a wrong or incomplete answer from it, and a system can retrieve an imperfect set of passages and still produce something usable. Both halves need to be evaluated, not just the second one, because it’s the one that’s visible in the output.

This is close to how we approached a RAG system built to compare day-to-day hospital process against a governing reference standard for a healthcare client. The value was in grounding the comparison in a single authorised document, not in the model’s general medical knowledge. More on that in our work.

Evaluation: a convincing demo is not a test suite

A demo that looks good to the people who built it is not an evaluation framework. It’s a small number of examples, usually chosen because they work well, reviewed by people who already know what a good answer looks like. That tells you very little about how the system behaves on the thousandth case, the odd one, or the one a new user phrases unexpectedly.

What’s needed instead is evaluation that reflects the actual business task: a representative set of real or realistic cases, criteria for correctness that map to what the business actually cares about (which is not always the same as fluency or plausibility), and a way to run that evaluation repeatedly as the system, the underlying data, or the model itself changes. What “good” means is genuinely specific to the task and the organisation’s risk tolerance. A universal accuracy threshold that applies equally to a drafting assistant and a decision-support system for regulated advice doesn’t reflect how differently those two systems fail. The evaluation criteria have to be built for the task in front of you, not borrowed wholesale from a vendor’s benchmark.

Guardrails and boundaries

A production AI system needs an explicit definition of what it is permitted to do, what it is not permitted to do, and what falls into a middle category that requires escalation to a person rather than an autonomous decision either way. In practice this tends to sort into three groups: actions the system can take on its own with confidence, actions it should never take regardless of how confident it appears, and actions that look routine most of the time but occasionally carry enough consequence that a human needs to be in the loop before anything happens.

Getting this boundary right requires understanding the actual downside of the system being wrong in a given context, not a generic sense of “AI safety.” A system drafting a customer email and a system approving a refund carry very different consequences for the same underlying mistake, and the guardrails around each should reflect that difference rather than apply one policy uniformly across both.

Human oversight, designed by risk

Human-in-the-loop is often added to an AI system as a vague disclaimer (“a human reviews the output”) without much thought given to which outputs, reviewed by whom, against what standard, and with what authority to intervene. That’s not oversight; it’s a compliance checkbox that happens to involve a person.

Real oversight is designed deliberately, proportional to consequence. Low-risk, easily reversible actions might reasonably run with light-touch or no human review. Higher-stakes or harder-to-reverse actions warrant a defined reviewer with the context and authority to catch a problem before it has an effect, not after. The design question is not a blanket “should there be a human in the loop” but a set of more specific ones for each consequential decision point: whether oversight is needed at all, where in the process it belongs, what the reviewer is actually checking, and what authority they have to stop or correct the system before it acts.

Integration: living inside the business, not beside it

An AI system that exists only as a standalone chat window, isolated from the workflow it’s meant to improve, can struggle to achieve broad operational adoption even when its underlying capability is strong. It depends on someone remembering to open it, having the discipline to check its output, and manually carrying the result back into whatever system of record actually runs the business. That extra friction is often enough on its own to keep a genuinely capable tool from ever being adopted at scale.

Enterprise AI earns its value by operating inside the systems and processes that already exist: triggered by the same events that already trigger work, reading from and writing to the same systems of record the business already relies on, and surfacing its output where the relevant person is already working rather than in a separate tab they have to remember to visit. That’s usually a larger share of the engineering effort than the AI component itself, and it’s the part that determines whether the system gets used at all.

Observability

Once a system is live, the organisation needs ongoing visibility into how it’s actually behaving, not just confirmation that it shipped. That means tracking failures and their patterns, output quality on an ongoing basis rather than only at launch, the cost of running the system at real volume, latency under real load, actual usage against what was expected, and whether behaviour is drifting as the underlying data, user base, or upstream model changes.

Without this, problems tend to surface first as a complaint from a business user rather than as a signal the engineering team caught early, which is usually the more expensive way to find out.

Production is a lifecycle, not a milestone

Deployment is not the finish line. A production AI system needs ongoing monitoring, periodic re-evaluation, and deliberate iteration as data, models, users and business conditions change under it, the same way any other production software does, with the added complication that the model’s behaviour itself can shift under conditions that look unchanged from the outside.

That ongoing discipline is as much an organisational capability as a technical one. Some of it is a matter of process and ownership; some of it depends on the organisation having people who understand both the business task and the system well enough to keep it accurate over time, which is part of why we also work with organisations on building that capability directly, through enterprise AI training, alongside the systems themselves.

None of this is a reason to avoid production AI. It’s a reason to treat “in production” as a standard to design for from the start, rather than a milestone a prototype eventually crosses on its own.