Ninety-five percent of enterprise generative-AI pilots in 2025 returned nothing you could put on a profit-and-loss statement. That is MIT's number, from its widely-cited "State of AI in Business" report, and it is the single most useful fact in this whole market — because an entire category of company, the AI consulting startup, was built to sell you the pilot that lands in that 95%. The pitch and the failure rate are not in tension. They are the same business model viewed from two angles.
After ChatGPT, the category exploded. Every boutique agency, every ex-McKinsey associate with a Notion page, every dev shop that had been doing WordPress in 2021 rebranded "AI-first" by Monday. They show up with the same shape of offer: an AI readiness assessment, a strategy workshop, an opportunity roadmap, and — if the budget is large enough — a flashy proof of concept. The deliverable is almost always a document and a demo. That is the tell, and it is the whole argument of this piece.
The consensus take, and why it is wrong
The conventional narrative goes like this: AI is moving too fast for your team to keep up, you lack an "AI strategy," and what you need before you spend real money is an expert to assess your readiness and chart a roadmap. It sounds responsible. It is mostly theater. The premise — that your problem is a knowledge gap that a slide deck closes — is the part nobody on the selling side will examine, because examining it ends the engagement.
You do not have an AI strategy problem. You have a software problem. The question was never "should we use AI" — that was answered for you the moment your competitors started shipping. The question is whether you can integrate a probabilistic, expensive, drifting model into systems that were designed for none of those things, and keep it working after the consultant's invoice clears. A roadmap does not answer that. Code that runs in production answers that.
The demo is free now — that is the whole problem
Here is what the AI consulting startup does not advertise: the proof of concept they charge five figures for is the cheapest thing in the building. Any competent engineer with an API key builds a convincing chatbot or document-summarizer demo in a weekend. The frontier models do the heavy lifting; the wrapper is thin. The deliverable that consulting firms have organized their pricing around is precisely the deliverable that the tools have driven to near-zero marginal cost.
A demo answers a question nobody serious is asking — "can the model produce a plausible output once, on a clean example, in front of an audience?" Of course it can. The questions that decide whether you join the 95% are the ones a demo is structured to hide: what happens on the malformed input, the adversarial user, the 10,000th request; what it costs per month at real volume; how you tell when it has quietly started being wrong. None of that fits in a workshop. All of it is engineering.
A strategy deck has a shelf life of one model release
There is a more specific reason to distrust the roadmap as a product: it decays faster than almost anything else you can buy. The tool you were advised to standardize on in the spring is deprecated by autumn; the architecture that made sense before long-context models or cheap function-calling is a liability after. At my own studio we run a quarterly review where the leadership team evaluates new tooling from scratch and rejects 60 to 80 percent of what we look at — not out of conservatism, but because most of it does not beat what we already run. A twelve-month AI roadmap sold as a fixed artifact is obsolete before the ink dries. The only durable strategy is a team that re-decides every quarter, which is not a thing you can hand over in a PDF.
Where the 95% actually dies
Walk back from the failures and you find the same graveyard every time, and it is never the model. It is retrieval that returns confident garbage because nobody cleaned the source data. It is an agent that works in the happy path and loops forever in the real one. It is a pipeline with no evaluation harness, so "it got worse" is a vibe nobody can measure. It is the integration into the legacy billing system that the demo politely skipped. It is the monthly token bill that quietly exceeds the salary of the person the tool replaced. This is the boring 90%, and it is the entire job.
This is the lens I cut the topic with. Our generative-AI work — RAG and AI search, agentic workflows, document processing, copilots — lives or dies on the unglamorous parts: vector stores that are tuned rather than defaulted, evaluation tracing with tools like Langfuse and Phoenix so regressions are caught instead of felt, and DevOps owned by the same engineers who wrote the application code rather than thrown over a wall to a separate department. Every pull request is reviewed by at least one other senior engineer before it merges. None of that is sellable as a workshop. All of it is the difference between a pilot and a system.
Advice leaves. Builders stay. The model is the message.
The structural problem with the AI consulting startup is encoded in its engagement shape: come in, assess, advise, present, leave. The incentive is to maximize billable insight per week and exit before the maintenance reality arrives. But AI systems are not a one-time install. They drift. The data changes, the model version changes, the user behavior changes, and a thing that worked in March is subtly broken by June. Whoever is not in the room in month nine is not the team that determines whether your project worked.
That is why I would weigh a firm's retention over its slide template every time. Ours runs under 5% engineer turnover a year against an industry norm north of 20%, with average engineer tenure around eight years and the average client engagement around four. That is not a culture brag — it is the operational fact that decides whether the people who understand your evals and your edge cases are still there when something breaks. A built generative-AI application that reads screenplays and composes visual scenes is not a deck; it is something that has to keep running, and someone has to still know how it works.
Four questions before you sign anything
Commit to a few questions and the category sorts itself fast. Ask who maintains the system in month nine, and watch whether the answer is a name or a renewal. Ask to see the evaluation setup — if there isn't one, you are buying a demo, not a product. Ask for the repository, not the presentation. Ask what they would refuse to build and why; a firm that has never said no to a feature has never protected a codebase. And price the asymmetry honestly: a fractional-CTO arrangement that ranges from roughly $4,000 to $16,000 a month buys ongoing engineering judgment, while a comparable strategy retainer often buys a document that expires before you finish reading it.
None of this means there is no role for outside help — there obviously is. It means the help you should pay for is the kind that ships and stays, not the kind that assesses and leaves. The good news, if you can call it that, is that the demo is now genuinely free; the hard part was always the part you can't put on a slide. Buy that part.
We've made the full argument in what AI consulting really is. When you want the version with code attached, our AI engineering services are where to look.
The bottom line for a startup buying AI help
If an AI consulting startup's core deliverable is a document, you are buying the 10% the tools already gave away for free, and you will likely finance your own entry into the 95% that returns nothing. The roadmap is not the work. The workshop is not the work. The demo is the least of the work. The work is integration, data, evaluation, cost control, and the unglamorous discipline of keeping a drifting system honest for years — and that is engineering, done by people who are still there when it breaks. Hire builders who stay. Treat anyone selling you AI strategy as a deliverable in itself as exactly what the 95% looks like before it shows up on the spreadsheet.