Most companies I talk to already have an AI prototype. It answered questions well in a demo, someone senior liked it, and six months later it still runs on a laptop or in a sandbox nobody touches. The reason is rarely the model: the prototype stalls on decisions that should have been made before the first prompt was written, and that are much harder to make once a demo exists and people expect a launch date.
How many AI projects stop after the proof of concept?
Enough that analysts forecast it. In July 2024 Gartner predicted that at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025, citing poor data quality, inadequate risk controls, escalating costs or unclear business value (Gartner, July 2024). In June 2025 it predicted that over 40% of agentic AI projects will be canceled by the end of 2027, for escalating costs, unclear business value or inadequate risk controls (Gartner, June 2025).
Both are predictions, not measured failure rates, so I would not quote either figure as a statistic. The list of reasons is the useful part. Costs, value, risk controls and data quality can all be checked before a pilot starts, and in my experience that is where a project's chances of reaching production are mostly decided.
Why does an AI prototype that works in a demo stall?
A demo proves the model can do the task on examples someone picked. Production asks different questions: better than what, on which cases, what happens when it is wrong, where the data goes, what it costs at full volume. A prototype built without answers to those questions has nothing to show the people who sign off on production. The table lists the stall patterns I see most often and the signal that gives each one away.
| Why it stalls | Signal you'll see | What fixes it |
|---|---|---|
| No baseline | "It works well" with no number to compare against | Measure the current process first: accuracy, time, cost |
| Scope too wide | The prototype handles everything a little, nothing reliably | Cut to the narrowest task that still matters |
| No plan for errors | Debate over whether the model is "good enough" to trust | Automate the clean cases, send exceptions to a person with a reason |
| Compliance left for later | Security review starts after the demo and stops the project | Decide where models and data run before writing code |
| Costs not measured | Nobody knows the cost per document, ticket or call | Log the cost of every AI call from the first version |
| Built away from real systems | Demo runs on an export; integration is "phase two" | Build the pilot in the environment it will run in |
What do AI projects that reach production have in common?
I have taken several AI systems to production: an LLM agent that routes more than 1 million help desk tickets a year at a major Italian bank, automated inheritance document checks, a financial statement extraction pipeline handling 2,500+ documents a day, a real-time voice agent for clinic reception, and the AI pipelines of a consumer finance platform. These very different systems shared a short list of decisions taken early, and each of the next five sections covers one of them.
Why measure a baseline before building the AI?
Without a baseline nobody can tell whether the system works. With one, the conversation with the steering committee becomes a comparison. On the bank help desk, people routing tickets by hand were right 75% of the time across 400+ categories. The LLM agent reached 81%, and average resolution time went from 5-6 days to 2-3, because fewer tickets bounced between the wrong teams. Those numbers are what moved the project past the pilot. Read the details in the help desk ticket routing case study.
If no baseline exists, measuring one is the first task. It usually takes a sample of a few hundred real cases and a person timing how long they take and how often they get them right. It is unglamorous work, and it decides whether an 81% result looks like a win or like a system that is wrong one time in five.
How narrow should the scope of a first AI project be?
Narrower than feels comfortable. The IGEA voice agent I built as co-founder and CTO answers clinic reception calls and handles bookings, changes and cancellations. It deliberately does no medical triage. That choice kept the agent predictable, and it is why the agent was fully operational with staff trained in 2 weeks at a clinic with 40+ doctors. In the first month it missed 0 calls and scored 96.7% on success.
A wider scope would have meant months of edge cases and a much harder conversation about risk. Once a narrow scope runs reliably, adding the next task is a smaller project than starting broad.
What should happen when the AI gets a case wrong?
Every model gets some cases wrong, so the question to settle before the pilot is what the system does with them. The pattern that worked in my document projects is to automate the cases that pass every check and send the rest to a person with the specific reason the check failed.
In inheritance document checks at a major Italian bank, documents are compared with the data the bank already holds. Standard files go ahead with no human step, and a failed check reaches an operator with the discrepancy named, so the operator reviews one point and not the whole file. Handling time fell by 80%. In the financial statement pipeline, about 30 validation checks run on every document (totals that must reconcile, for example), and manual work fell by 84%. In both cases reliability comes from the checks, which is also what lets a risk committee approve the system.
When should you decide where the data and the models run?
Before any code is written. In my banking projects the models ran on Azure OpenAI inside the client's EU cloud, sensitive data never went into logs and every production document had an audit trail. Some pipelines ran entirely on the bank's own OpenShift. None of that can be bolted on after a demo built on a public API with production data in the prompts.
Enterprise integrations take longer than a prototype because of security and compliance reviews. That time is part of the project, so plan for it in the schedule. A pilot that has already been designed for the approved cloud reaches that review with most of the answers ready.
Why log the cost of every AI call from the first version?
A prototype that costs a few cents per request looks cheap until you multiply it by real volume. At 1 million tickets a year or 2,500 documents a day, small inefficiencies become a budget line someone will question. When I took over the AI pipelines of a consumer finance platform, they made more model calls than needed, sent oversized context and had no ceiling on spend. Removing redundant calls, trimming context, caching repeated answers and adding spend caps and rate limits cut AI costs by about 80% with the same features. Every AI-generated report now logs its own cost.
If the cost per unit is logged from the first version, the business case for production is a calculation anyone can check. Without it, cost becomes one of the reasons a project is stopped. The cost reduction case study shows what changed.
Which first AI project has the best chance of reaching production?
Internal processes with high volume and a measurable baseline: routing, document checks, data extraction. They use data that already sits inside the company's perimeter, mistakes reach an employee before a customer, and results are easy to compare with the previous process. A customer-facing project can work too, as the clinic voice agent shows, but only with a scope narrow enough to be reliable on day one.
As a first step I would avoid the typical request for an assistant that knows everything about the company. Nothing gives you a baseline for everything, and nobody can say when such a project is done.
What should you check before starting an AI pilot?
| Question | Good answer before you start |
|---|---|
| What does the system do, in one sentence? | One task with clear inputs and outputs |
| How is that task done today, and how well? | A measured baseline: accuracy, time per case, cost |
| What result makes it worth going to production? | A target agreed with the business owner, written down |
| What happens to the cases the AI gets wrong? | Checks, a review queue, a person with the reason |
| Where do the models and the data run? | Cloud or on-premise environment approved by security |
| What does one case cost, and at full volume? | Cost per call logged, spend caps planned |
| Who owns the system after launch? | A named team that will run it and watch the numbers |
What are the common mistakes when moving an AI pilot to production?
The first is judging the pilot on a demo instead of a sample of real cases, including the messy ones. The second is reporting average accuracy without saying what happens to the errors; a risk committee will ask where the wrong 19% goes. The third is starting the security review after the pilot, when the architecture is already set. Another is letting the scope grow during the pilot because the first results look good. The last is treating costs as a later problem: the pilot is the cheapest moment to measure them.
Have an AI prototype that never left the pilot?
Tell me what it does and where it stopped. I'll tell you what is missing to put it in production, and whether it is worth doing.
Let's talkIf you work in a bank or a financial services company, the page on AI consulting for banks and financial services describes how I pick and deliver first use cases. For agents in other sectors, see AI agent development.
