The Death of Shelf Software
and the Rise of Composable AI

If your organisation has bought AI software only to watch staff spend more time correcting it than using it, you're in miserable company—but you're not alone. Welcome to the AI value gap, where the chasm between promise and performance has become an industry-wide crisis.

The Widening Chasm

BCG's September 2025 study of more than 1,250 firms worldwide delivers a sobering verdict. They categorise organisations into three tiers: the "future-built" (5%), the "scalers" (35%), and the "laggards" (60%). The gap between these groups isn't just wide—it's accelerating.

"The gap between AI hype and AI reality has become a chasm. Most enterprises are dabbling, stalling, or scaling noise."

The Abandonment Crisis

S&P Global's research paints an even grimmer picture. The share of companies abandoning most of their AI initiatives jumped to 42% in 2025—up from just 17% the previous year. This isn't a marginal miss. This is a 25-percentage-point surge in failure rates year-over-year.

These aren't pilots that failed in the lab. These are initiatives that consumed budget, management attention, and political capital, only to be quietly written off. The average organisation scrapped 46% of AI proof-of-concepts before they reached production.

The Regulatory Alarm Bell

When Gartner warned over a year ago that the majority of generative AI pilots would die after proof-of-concept, many executives dismissed it as typical analyst pessimism. Their prediction has aged like fine wine.

But the most telling signal isn't from consultancies—it's from regulators. The FTC and SEC have moved from investigation to prosecution, filing enforcement actions against companies engaging in "AI-washing"—making deceptive claims about AI capabilities that don't exist.

The Human Cost

The productivity paradox is the cruelest twist. The promise was automation and efficiency. The reality in many organisations is staff spending more time checking AI-generated work than it would have taken to do the work themselves.

Every output needs review. Every recommendation needs verification. Every automation needs a human safety net. The AI was supposed to multiply productivity. Instead it's become a productivity tax.

"We hired the AI to save time. Now we hire people to babysit the AI."

EY's 2025 survey found that most companies have suffered some form of risk-related financial loss from deploying AI. Not "might suffer" or "could potentially face." Have suffered. Past tense. Real money. Real compliance violations. Real customer-facing errors that required expensive remediation.

The Root Cause Question

So what's driving this spectacular failure rate? Is AI fundamentally oversold? Are the models not ready? Are enterprises just "doing it wrong"?

The answer is more structural—and more fixable—than either AI evangelists or sceptics want to admit. The crisis isn't technical. It's procurement, architecture, and organisational design colliding with a technology that doesn't behave like the enterprise software that came before it.

Two forces are at work simultaneously:

1. Enterprises are running a 2005 procurement playbook against 2025 technology. Requirements-driven RFPs, feature checklists, multi-year contracts with vendor lock-in—all optimised for stable, deterministic software. AI is neither stable nor deterministic.
2. Many vendors are selling yesterday's software with today's buzzwords. Legacy products with "AI" badges slapped on. Models baked in 18 months ago, now two generations behind. Proprietary wrappers around commodity APIs. The FTC is prosecuting this behaviour, but the market incentives remain misaligned.

The organisations in the 5% aren't smarter. They're not luckier. They didn't buy better AI. They bought differently, and they built differently. That difference—between 95% failure and 5% success—is what the next fifteen chapters unpack.

Traditional software procurement worked because the underlying assumptions held true: requirements were knowable upfront, vendor products were mature and stable, and implementation timelines measured in quarters were acceptable. The playbook was simple, effective, and catastrophically mismatched to AI.

The Old Playbook That Still Dominates

Walk into any enterprise procurement department and you'll see the same ritual playing out:

1. Gather requirements from stakeholders across the organisation
2. Write a detailed specification document (50-200 pages)
3. Issue an RFP to three shortlisted vendors
4. Evaluate demos against your checklist over 3-6 months
5. Negotiate a multi-year contract with service-level agreements
6. Deploy over 6–18 months with a systems integrator

The software would do roughly what the specification promised, or at least close enough that the delta could be managed with training and process adjustments. This worked for decades.

This model made perfect sense for ERP systems, CRM platforms, and business intelligence tools. Requirements for payroll processing or inventory management don't change fundamentally year-to-year. Oracle or SAP could freeze a scope, build to spec, and the product would remain relevant for a 3-5 year contract lifecycle.

"We spent nine months on the RFP. By the time we selected a vendor, the AI landscape had shifted twice. The capabilities we specified no longer existed."

Why AI Shatters Every Assumption

AI doesn't just bend the old procurement model—it breaks it completely. Here's why:

The Time Paradox

Traditional RFP timelines are measured in quarters. AI evolution is measured in weeks.

The Outcome: Predictable Failure

When you run the old RFP playbook against AI, you get exactly the outcomes we're seeing:

• 42% abandonment rates (S&P Global, 2025) — Projects that consumed months of requirements gathering, vendor evaluation, and contract negotiation, only to be quietly shelved because the deployed system doesn't match the evolved needs
• Productivity regressions — Staff spending more time checking AI outputs than doing the work themselves, because the RFP specified "AI-powered" but didn't specify "actually useful"
• Expensive write-offs — Three-year contracts signed for capabilities that became obsolete in six months, with no upgrade path that doesn't require renegotiating the entire deal

Not because AI doesn't work. Not because the vendors are incompetent. But because procurement assumed the technology would behave like the ERP systems it replaced.

It doesn't.

"We ran a textbook procurement. By-the-book RFP, thorough evaluation, robust contract. And it delivered exactly what we asked for—which turned out to be completely wrong."

The Fundamental Mismatch

The core problem is categorical. Traditional procurement treats software as a product to install. AI demands you treat it as a capability to cultivate.

The organisation that tries to RFP its way to AI maturity is bringing a requirements document to an experimentation contest. It's bringing a five-year roadmap to a quarterly refresh cycle. It's bringing install-and-forget expectations to a tune-and-measure reality.

And then wondering why 42% of projects get abandoned.

The procurement mismatch is only half the equation. The other half is what many vendors are actually selling when they slap "AI-powered" on the product brochure. When regulators start filing enforcement actions, you know the problem has become systematic.

The Existential Vendor Dilemma

Legacy enterprise software vendors faced an existential threat in 2023: get AI into the product or watch customers churn to startups promising 10× productivity. So they did what large organisations do under pressure—they rushed.

Most started integrating AI 12–18 months ago, when GPT-3.5 was state-of-the-art and patterns for production deployment were immature. By the time their products ship today through the typical 9–12 month enterprise software development cycle, the "AI" baked into the product is already two generations behind what dominates the press and AI-native startup demos.

The Rebranding Strategy

Worse, some vendors didn't even bother integrating new models—they just rebranded existing features:

"The autocomplete we've had for five years? Now it's a 'Generative AI Writing Assistant.' Same code. New badge."

FTC Enforcement: Case Studies

The Incentive Structure Problem

When regulators move from investigation to prosecution, it signals the market incentives have become deeply misaligned. Vendors get rewarded for claiming AI capabilities, not for delivering AI value.

• Sales cycles close faster when the deck has "AI" on every slide
• Renewals are easier when you can point to an "AI roadmap"
• Stock prices respond to AI announcements, not AI performance
• Competitive pressure forces "me too" announcements

The market is buying the promise, and many vendors are optimising for the promise instead of the performance.

The Black Box Business Model

Even vendors trying to do it right are constrained by their own business model. They can't give customers what would actually create value because it would undermine their competitive position:

The Commodity Trap

Everyone has access to the same OpenAI or Anthropic APIs. The only thing separating one vendor's "AI solution" from another's is:

1. The wrapper — UI and integration glue
2. The integration tax — How much work to connect to your systems
3. The sales story — Marketing narrative and brand trust

None of these create durable competitive advantage when models are perishable and customers are learning that "AI-powered" is often just brand paint.

"We pay $200k/year for 'enterprise AI.' Last week I built the same thing in an afternoon using the OpenAI API and $50 of credits."

The Predictable Outcome

When you buy "AI software" from an established vendor, you're often getting:

• ✗ Yesterday's model (2-3 generations behind)
• ✗ In a proprietary wrapper (no model swapping)
• ✗ With no upgrade path (locked into obsolescence)
• ✗ Evaluated against cherry-picked demos (not your data)
• ✗ Sold by a team optimised for deal closure (not deployment success)

And then everyone acts surprised when 42% of projects get abandoned.

To understand why shelf software is dying, you need to understand why it existed in the first place. The entire history of commercial software is a history of cost amortisation—and AI just made amortisation irrelevant.

The Bespoke Era (1970s–1990s)

In the beginning, all business software was bespoke. If you wanted a system to manage inventory or process payroll, you hired a team of developers to build it from scratch.

The price tag put bespoke software out of reach for most organisations, but for those who could afford it, the perfect fit was worth the premium.

The Shelf Revolution (1990s–2010s)

Then someone had a clever insight: What if we built the software once and sold it to 1,000 customers?

This was the birth of the software industry as we know it. Oracle, SAP, Microsoft, Salesforce—all built on this model. Develop once, sell repeatedly, amortise the fixed costs across a growing customer base.

The compromise was explicit and acceptable:

The Cloud Evolution (2010s–2020s)

Cloud and SaaS took amortisation even further. Multi-tenancy meant one instance of the software served thousands of customers simultaneously, reducing infrastructure costs. Subscription pricing smoothed revenue and lowered the upfront barrier. Implementation times dropped from quarters to weeks.

But the fundamental model remained: one product, many customers, amortised development costs.

"The economics still made sense because software development was expensive. Even with agile and modern tooling, building production-grade enterprise apps cost millions."

The AI Inflection Point (2025+)

AI hasn't just improved software development—it's collapsed the cost structure entirely.

What used to take a team of six developers eight months can now be generated by AI in days or weeks. Marginal cost of software creation is approaching zero.

The Go-to-Market Tax

Here's what customers actually pay for when they buy shelf software:

AI Eliminates the Sales Process

Traditional procurement takes 3–9 months because:

• Requirements gathering is labour-intensive
• Vendor evaluation requires extensive demos
• Contract negotiation is adversarial and time-consuming

AI collapses this entire cycle.

"When technology can discover requirements and generate solutions faster than a sales engineer can schedule a demo call, the traditional vendor model collapses."

The Shelf Software Compromise

For decades, customers accepted these compromises because shelf software was the only affordable option:

This compromise made economic sense when the alternative was $2 million in bespoke development costs.

It makes zero sense when AI can deliver 100% fit for a fraction of the shelf subscription price.

The Inflection Point

We've crossed a threshold where three conditions are simultaneously true for the first time in software history:

If Chapter 4 explained why shelf software is dying economically, this chapter is about what happens when those economics actually shift in the market. Spoiler: the shift is already underway.

The Value Proposition That No Longer Holds

Off-the-shelf software survived for decades on a simple value proposition:

"It's not perfect, but it's affordable and fast to deploy."

Both parts of that proposition are now false for AI applications.

Distribution Was the Moat—And AI Just Drained It

The shelf software industry built its competitive advantage on distribution, not technology.

The "Safe" Choice Is No Longer Safe

What shelf vendors really sold was risk reduction: "Nobody ever got fired for buying Oracle."

That risk reduction came from:

• Brand trust — decades of market presence
• Market share — "everyone else uses it"
• Conventional wisdom — the implicit safety of standard decisions

"Playing it safe by buying Oracle is how you end up with a 42% abandonment rate. The safe choice became the risky one."

Composition vs Installation

The organisations getting value from AI aren't the ones buying shrink-wrapped "AI solutions" from legacy vendors.

They're the ones treating AI as a capability to compose, not a product to install.

That compositional approach is fundamentally incompatible with the shelf software model. You can't compose what's locked in a vendor's proprietary wrapper.

Why Shelf Software Is Dying

Shelf software isn't dying because it's badly built. Oracle's database technology is excellent. Salesforce's CRM is feature-rich. SAP's ERP handles complex processes.

Shelf software is dying because the economic conditions that made it rational no longer exist.

AI models have a shelf life measured in months, not years. This creates an existential problem for traditional software contracts that lock organisations into technology that will be obsolete before the ink dries.

The Model Generation Gap

GPT-3.5 launched in March 2022 to widespread amazement. It could hold conversations, write code, and summarise documents with unprecedented fluency. Organisations that moved quickly signed contracts in Q2 2022, locking in what felt like cutting-edge capability.

By March 2023—exactly one year later—GPT-5 made GPT-3.5 look like a toy. Better reasoning across complex tasks. Context windows expanded from 4,096 tokens to 32,768 tokens, allowing it to process entire codebases or lengthy legal documents. Dramatically more reliable outputs with fewer hallucinations. GPT-3.5 went from "state of the art" to "legacy technology" in 365 days.

But the evolution didn't stop there. By November 2023, GPT-5 Turbo arrived with updated knowledge cutoffs and reduced pricing. Throughout 2024, Claude Sonnet 4.5, Gemini Pro 3, and rapidly-improving open models like Llama 3 pushed the frontier even further. By late 2025, GPT-5, Claude Sonnet 4.5, and Gemini Pro 3 are all shipping, while open models iterate at speeds that make proprietary vendors nervous.

The capability you bought in 2023 is now perishable goods, but your three-year contract treats it like durable infrastructure.

"We're contractually obligated to use a model that's three generations old. Our competitors upgraded six months ago. We can't, unless we renegotiate the entire deal."

Traditional Software Aged Gracefully

This perishability problem is unique to AI. Traditional enterprise software aged slowly and predictably. SAP ERP from 2015 is still fundamentally useful in 2025—the core business logic for managing inventory, processing payroll, or tracking orders hasn't changed. Customer requirements don't fundamentally shift. Accounting principles remain stable. You could sign a five-year contract and expect reasonable value throughout the lifecycle.

Even SaaS platforms like Salesforce or Workday operate on annual or quarterly update cycles, with changes that are largely additive. The CRM you deployed in 2020 still performs its core function in 2025. New features arrive, but the foundation remains solid.

Perishability Isn't Just Performance—It's Economics

The cruelest twist: newer AI models aren't just better—they're often cheaper. GPT-5 is both more capable and more cost-effective than GPT-5 Turbo on a per-token basis. Claude Sonnet 4.5 offers superior reasoning at competitive pricing. Open models like Llama 3.3 deliver strong performance with zero API costs if you self-host.

When newer models offer superior quality at lower unit cost, sticking with an older model isn't just a capability gap—it's an economic penalty. You're paying more for worse results, not because you chose poorly, but because you chose once and couldn't adapt.

This creates a compounding disadvantage. Not only are you delivering inferior service (lower accuracy, slower processing, more errors), you're also paying more to deliver it. Your competitors who can swap models quarterly are simultaneously improving quality and reducing costs. You're stuck in a contract that optimises for neither.

Vendors Build Lock-In Mechanisms By Design

The vendors know this dynamic, which is why many are building proprietary wrappers and lock-in mechanisms. They can't compete on the underlying model—everyone has API access to the same LLMs from OpenAI, Anthropic, or Google. So they compete on making it difficult to leave.

Lock-in takes several forms:

• Proprietary data formats: Your fine-tuning data, evaluation sets, and prompt libraries are stored in vendor-specific formats that don't export cleanly
• Custom integrations: The vendor builds connectors to your CRM, ERP, and internal systems using their proprietary APIs, creating switching costs measured in engineering-months
• Black-box orchestration: How prompts are constructed, how outputs are post-processed, and how confidence scores are calculated remain opaque, making it impossible to replicate the workflow elsewhere
• Long implementation cycles: Nine-month deployments with extensive customisation mean switching vendors requires re-implementing everything from scratch

These aren't bugs—they're business model features. When the underlying technology commoditises rapidly (everyone can access GPT-5 or Claude Sonnet 4.5), the only defensible position is making migration painful.

"The vendor kept saying 'we use best-in-class AI models.' What they meant was 'we lock you to whichever model we chose, and upgrading requires renegotiating your contract.'"

Perishability Cuts Both Ways

But perishability creates an opportunity, not just a trap. When models improve rapidly, the value of vendor lock-in decreases. Why pay for a three-year contract with complex migration penalties when the technology you're locked into will be obsolete in six months anyway?

Smart organisations are inverting the dynamic. Instead of signing long-term contracts that assume stability, they're structuring procurement and architecture for continuous renewal:

This isn't optional sophistication—it's table stakes for avoiding the 42% abandonment rate. When your model strategy assumes continuous evolution, you're positioned to capture improvements rather than being trapped by obsolescence.

The Five-Year Illusion

Traditional software vendors thrived on five-year enterprise agreements because switching costs were genuinely high. Migrating from Oracle to SAP meant re-implementing business processes, retraining staff, and risking operational disruption. The cost of staying put was lower than the cost of moving.

AI inverts this calculus. The cost of staying put—being locked to obsolete models with inferior performance and higher costs—now exceeds the cost of moving, if you've architected for swappability. The organisations structuring their AI procurement for continuous renewal aren't being agile for agility's sake. They're acknowledging that perishability is the new normal.

The winning organisations are those that treat AI models like fresh produce, not like infrastructure. You don't sign a three-year contract for strawberries in January and expect them to be fresh in December. Why would you do that with AI models?

Traditional software procurement relied on feature checklists. You'd list 50 required capabilities, 30 nice-to-haves, and 20 deal-breakers. Vendors would demo their way down the list, checking boxes until someone won the RFP. For AI, this approach isn't just ineffective—it's actively misleading.

How Feature Checklists Used to Work

The feature checklist model made perfect sense when features were deterministic and testable. Either the software can export to CSV or it can't. Either it supports single sign-on with Active Directory or it doesn't. Either the CRM integrates with Salesforce or it requires manual data entry. Binary, verifiable, contract-able.

Procurement teams would create exhaustive lists:

• ✓ Multi-factor authentication
• ✓ Role-based access control
• ✓ RESTful API with OAuth 2.0
• ✓ SOC 2 Type II compliance
• ✓ 99.9% uptime SLA

Vendors would respond with "yes," "no," or "roadmap Q3 2024." You could score responses objectively, compare vendors side-by-side, and make procurement decisions based on coverage percentage. The vendor covering 90% of requirements beats the one covering 75%. Simple math.

AI Capabilities Don't Work That Way

Now consider AI features. "Can it summarise documents?" On the surface, this seems like a reasonable yes/no question. But the real answer is: yes, but summarise well or poorly? For what kinds of documents? With what failure modes?

Every AI capability is probabilistic and context-dependent. Saying "we have AI-powered document summarisation" is like saying "we have transportation"—technically true but meaninglessly vague. A bicycle and a helicopter both provide transportation, but you wouldn't choose between them based on a feature checklist that asks "Does it move people from point A to point B?"

"The vendor demo showed perfect document summarisation. In production, it hallucinated citations, missed critical clauses, and required more review time than just reading the document."

The Questions Behind the Features

What matters isn't whether the feature exists, but whether the capability performs reliably on your tasks with your data in your workflows. And the only way to know that is through empirical evaluation, not vendor demonstration.

How Feature Lists Enable Deception

Feature checklists let vendors show demos with cherry-picked examples. They can demonstrate AI document summarisation on a clean, well-structured 10-page PDF with straightforward content. What they won't show you:

• Performance on 200-page scanned contracts with tables, appendices, and marginalia
• Behaviour when critical information appears in footnotes or image-embedded text
• Accuracy on domain-specific jargon your industry uses but wasn't in the training data
• Failure modes when documents contain conflicting information or ambiguous phrasing
• Degradation on edge cases like multi-language documents or non-standard formatting

The demo shows capability existence. It reveals nothing about production performance, which is the only thing that matters.

What Smart Buyers Demand Instead

Smart buyers are shifting from feature checklists to performance audits. They're treating vendor claims as hypotheses to test, not facts to accept. Here's what the new procurement looks like:

This isn't about being difficult—it's about demanding evidence that the capability actually works for your use case. Vendors who can't provide this documentation aren't selling AI capabilities; they're selling promises wrapped in feature lists.

The Performance Audit Mindset

The shift from features to performance requires a fundamental change in how procurement operates. Instead of asking "Does your product have feature X?" you're asking "Can you prove feature X works on tasks like ours?"

This means:

• Empirical testing replaces vendor demos: You run the AI on your data, not watch the vendor run it on theirs
• Metrics replace checkboxes: Precision, recall, F1 scores, cost per request, time savings measured in hours—not "yes/no" on feature lists
• Failure modes get disclosed upfront: "Works 95% of the time on X, performs poorly on Y" beats "Works on everything!" with hidden gotchas
• Continuous evaluation replaces one-time verification: You're auditing ongoing performance, not just checking initial compliance

"We stopped asking 'Do you have this feature?' and started asking 'Can you prove it works on our data?' Half the vendors disappeared. The ones who stayed were the ones worth evaluating."

Why This Matters More Than Ever

As AI capabilities become table stakes, feature checklists become even less meaningful. When every vendor claims "AI-powered search," "AI-driven insights," and "machine learning optimization," the checklist becomes a list of identical claims with zero differentiation.

Performance evaluation is the only trustworthy signal. It forces vendors to prove their claims on your tasks, with your data, under your constraints. It reveals the gap between marketing promises and production reality before you sign a multi-year contract.

When features are probabilistic and context-dependent, the only trustworthy evaluation is empirical performance on representative tasks. Anything else is AI theater dressed up as procurement.

If shelf software is dying and pure bespoke drowns you in maintenance costs, what's the alternative? The answer is composable bespoke on a stable platform—a three-layer architecture that lets you buy commodity infrastructure, abstract perishable models, and generate task-specific applications that live only as long as they provide value.

The Three-Layer Model

Think of AI architecture as three distinct layers, each with different economics, lifecycles, and procurement strategies. Getting this separation right is the difference between flexibility and lock-in, between continuous improvement and planned obsolescence.

Layer 1: Buy the Foundation (Durable Infrastructure)

Some capabilities benefit massively from economies of scale and don't differentiate your business. You're not competing on who has better identity management or more sophisticated logging. Buy these as managed services and move on.

What belongs in Layer 1:

• Identity and access management: Auth0, Okta, Azure AD, AWS Cognito—you're not competing on login flows. Buy mature, compliant IAM and forget about it.
• Data infrastructure: Managed databases (Postgres, MySQL), data warehouses (Snowflake, BigQuery), vector stores (Pinecone, Weaviate, Qdrant). Commodity infrastructure that benefits from vendor scale.
• Observability and monitoring: Datadog, Grafana, New Relic, Splunk—logging, monitoring, tracing, alerting. Buy mature stacks with integrations to your existing tools.
• Security and policy enforcement: Encryption, key management, secrets rotation, compliance controls—AWS KMS, HashiCorp Vault, cloud-native security services.
• CI/CD and deployment: GitHub Actions, GitLab CI, CircleCI, your existing DevOps tooling. Don't reinvent deployment pipelines.

This is the platform. It's boring, it's mature, and everyone uses roughly the same stack. The competitive differentiation comes from what you build on top of it, not from reinventing auth or monitoring.

Layer 2: Assemble with Swappable Adapters (Model Abstraction)

Don't hardcode model dependencies into your application logic. Use adapters that let you swap GPT-5 for Claude Sonnet 4.5 for Llama 3.3 without rewriting code. This is the layer that acknowledges perishability and gives you an upgrade path.

What belongs in Layer 2:

• Model routing layer: Abstract the LLM API so changing providers is a config change. Example: instead of calling openai.ChatCompletion.create() directly, call llm.complete() where llm is configured externally.
• Prompt management system: Store prompts separately from code, version them in git, A/B test different formulations. When you find a better prompt, deploy it without code changes.
• Evaluation harnesses: Test any model against your gold evaluation data before deploying it. Compare GPT-5 vs Claude Sonnet 4.5 vs Llama on your tasks, pick the winner based on metrics.
• Fallback chains and routing logic: Try GPT-5 first. On timeout or low confidence, fall back to Claude Sonnet. On ambiguous cases, escalate to human review. Route intelligently based on task complexity and model strengths.
• Cost and performance tracking: Log every model call with cost, latency, confidence score, and outcome. Know which models work best for which tasks, and what they cost per request.

This is your model abstraction layer. It acknowledges that models are perishable and gives you an upgrade path that doesn't require vendor renegotiation or application rewrites. When GPT-6 launches or a fine-tuned Llama model outperforms commercial APIs, you can swap it in and measure the improvement empirically.

"We spent two weeks building a model abstraction layer. It's saved us six months of rewrites across three major model upgrades."

Layer 3: Compose Task-Specific Micro-Apps (Ephemeral Applications)

This is where the magic happens. Generate small, focused applications that do one thing well, integrate with your platform (Layer 1), run on swappable models (Layer 2), and live only as long as they provide value.

Examples of Layer 3 applications:

These applications are ephemeral by design. They're cheap to build because AI assists in code generation. They run on a stable platform (Layer 1) so infrastructure is reliable. They can swap models freely (Layer 2) so they stay current as capabilities improve. When requirements change or better models arrive, you regenerate or retire them without disrupting the foundation.

Why This Model Wins

Composable bespoke architecture delivers five critical advantages that neither shelf software nor pure bespoke can match:

This Isn't "Build Everything Yourself"

Composable architecture is often misunderstood as "build everything from scratch." It's not. You're buying heavily for Layer 1 (infrastructure), buying or building thin adapters for Layer 2 (model abstraction), and generating Layer 3 (applications) with AI assistance.

The distinction matters:

• Traditional bespoke: Build your own database, auth system, monitoring, everything. Maintenance nightmare, high ongoing costs.
• Composable bespoke: Buy infrastructure that benefits from scale (Layer 1), build thin abstraction layers (Layer 2), generate task-specific apps (Layer 3). Maintenance focused on business logic, not plumbing.

You're not reinventing identity management or database engines. You're buying the boring infrastructure and focusing your engineering effort on the applications that differentiate your business.

"We buy the infrastructure that benefits from scale. We generate the applications that need to fit perfectly. That's the strategy."

The Path Forward

Composable architecture isn't a distant future vision—it's available today with existing tools. Cloud infrastructure is commodity. Model APIs are accessible. Code generation with AI assistance is production-ready. The organisations in the 5% aren't waiting for better technology—they're using what exists to build differently.

The shift is from "buy a product that almost fits" to "buy infrastructure, generate apps that fit exactly, swap components as technology improves." That's how you avoid the perishability trap, the lock-in spiral, and the 42% abandonment rate.

Traditional procurement starts with requirements: "We need software that does X, Y, and Z." Then you find a vendor who claims to do X, Y, and Z, negotiate a contract, and deploy. The assumption is that if the software meets the spec, it will deliver value. For AI, this is backwards. You don't know what will deliver value until you test it empirically.

The Requirements-First Model Breaks Down

The traditional model assumes you can specify outcomes before you test capability. Write down what you need, vendors bid on delivering it, you verify delivery against the spec. This works when capabilities are knowable and stable.

But AI capabilities aren't knowable until you test them on your data, in your workflows, with your edge cases. You can't specify "AI shall categorise support tickets with 90% accuracy" without first knowing whether 90% is achievable, what it costs to get there, and whether the remaining 10% creates unacceptable failure modes.

Hypothesis-Driven Procurement: The Framework

Hypothesis-driven procurement flips the model. Instead of gathering requirements, you formulate testable hypotheses. Instead of RFPs and demos, you run experiments. Instead of long contracts, you fund in stages based on empirical results.

Here's how it works:

Step 1: Define the Hypothesis

Instead of "We need AI-powered customer support," formulate a testable claim with three components:

1. Baseline: Current performance you're trying to improve (e.g., "4-hour average response time, manual categorisation taking 15 minutes per ticket")
2. Target delta: Specific improvement you're hypothesising (e.g., "85% categorisation accuracy, 60% response acceptance rate, 1-hour response time")
3. Measurable outcomes: Metrics you can verify empirically (accuracy, acceptance rate, time-to-response, cost per transaction)

Example hypothesis:

"An LLM can categorise incoming support tickets with 85% accuracy and suggest responses that agents accept without modification 60% of the time, reducing average response time from 4 hours to 1 hour."

This hypothesis is falsifiable. You can test it, measure the results, and either confirm or reject it based on data. It's not a requirement—it's a claim about what might be possible.

Step 2: Build an Evaluation Harness

Before anything touches production, create a controlled test environment. This is where most organisations skip steps and pay for it later.

What you need:

• Gold data: 200–500 historical examples with ground-truth labels. For support tickets, this means tickets with confirmed correct categories and ideal responses.
• Offline scoring: Run the AI on gold data without human involvement. Measure precision (how many AI predictions are correct), recall (how many correct cases does AI catch), F1 score (harmonic mean of precision and recall).
• Failure analysis: Identify systematic failure modes. Does it always miscategorise billing issues? Does it fail on tickets longer than 500 words? Does it struggle with non-native English?

Step 3: Shadow Deployment

If offline testing succeeds, run the AI in parallel with current processes without changing workflows. This reveals real-world complexity that gold data missed.

How shadow mode works:

• AI categorises tickets and suggests responses, but outputs are logged, not acted on
• Humans perform the work as usual—no disruption to operations
• Log where AI matches human decisions, where it diverges, and where humans would override it
• Run for 2-4 weeks to capture edge cases and drift patterns

Shadow mode reveals failure modes that offline testing missed. Maybe the AI performs well on historical data but poorly on recent tickets about a new product. Maybe accuracy is great overall but catastrophically bad on a specific ticket category that's high-stakes. You discover these issues in shadow mode, not production.

"Shadow deployment saved us. The AI looked perfect in offline testing. In shadow mode, we discovered it failed spectacularly on tickets mentioning our new product—which wasn't in the training data."

Step 4: Bounded Production Rollout

If shadow testing confirms the hypothesis, deploy to a limited scope where failure is contained and learning is fast:

• Route only low-severity tickets through AI (avoid high-stakes cases until confidence is proven)
• Enable AI suggestions for one product category (test with constrained complexity before expanding)
• Let 10% of agents opt into AI-assisted workflows (early adopters who'll provide detailed feedback)
• Measure relentlessly: Did response time actually drop to 1 hour? Is acceptance rate holding at 60%? Are there unexpected costs (more escalations, lower customer satisfaction, higher review time)?

Bounded rollout lets you capture most of the learning while limiting blast radius. If something goes wrong, it affects 10% of tickets, not 100%. If unit economics don't work at this scale, you kill before full deployment.

Step 5: Kill or Scale Based on Unit Economics

After 30-90 days of bounded rollout, you have data. Now make the binary decision: kill or scale.

Scale if:

• ✓ Performance hits target improvements (85% accuracy, 60% acceptance, 1-hour response time)
• ✓ Unit economics are positive (value > cost by at least 2×)
• ✓ Quality metrics remain stable or improve (CSAT, error rate, escalations)
• ✓ Agent adoption is strong (high usage, positive feedback)

Kill if:

• ✗ Performance misses targets by >15% (accuracy 70% when you need 85%)
• ✗ Unit economics don't work (cost > value, or ROI < 2×)
• ✗ Quality issues emerge (rising error rate, falling CSAT, high escalations)
• ✗ Agent adoption is low despite training (workarounds, complaints, non-usage)

The kill decision is critical. Don't fall into sunk cost fallacy. Don't "give it more time" if data says it's not working. Document why you're killing it so you don't repeat the mistake, then move on to the next hypothesis.

Why This Works: Rigour Replaces Hope

Hypothesis-driven procurement forces rigour at every stage. It prevents "deploy and hope." It makes failure cheap—kill after offline testing (costs: $5k and 2 weeks) rather than after 6 months in production (costs: $500k and organisational trauma).

This is how the 5% get consistent AI value. Not by buying better products—by testing hypotheses faster and killing failures earlier. Not by perfect requirements gathering—by empirical experimentation with clear success criteria.

The framework aligns procurement with the scientific method. You're testing claims with progressively higher fidelity, killing bad hypotheses early, and scaling only what's been proven empirically. That's the difference between the 5% who win and the 42% who abandon.

Feature checklists are dead (Chapter 7), but you still need to evaluate vendors and technologies. The replacement is capability audits—rigorous examination of whether the system can actually perform on your tasks, with your data, under your constraints.

From Feature Lists to Performance Verification

Traditional procurement asked "Do you have feature X?" Capability audits ask "Can you prove feature X works for us?" This isn't semantic nitpicking—it's the difference between vendor promises and production reality.

Here's what to audit instead of chasing feature lists:

Dimension 1: Data Access and Quality

Many AI solutions work beautifully on vendor-curated test data and fail spectacularly on messy enterprise reality. The first audit question: Can the vendor's AI work with your actual data?

Questions to ask:

• Data format compatibility: Will it handle your legacy PDFs, scanned images, semi-structured CSVs, or does it require perfectly formatted JSON?
• Data residency: Does the AI require data to leave your environment? If you're in healthcare or finance, can it run on-premises or in your VPC?
• Data preparation tax: How much cleaning and transformation is required before the AI can process your data? If it's "weeks of ETL work," factor that into cost and timeline.
• Volume handling: Demos show 100 examples. Can it handle your 10 million records without performance degradation?

Dimension 2: Evaluation Methodology

Can the vendor show you offline evaluation results on tasks similar to yours, with failure modes disclosed? If they can't or won't, that's a red flag—either they don't have the data or the results are bad.

What to demand:

• Precision and recall on representative tasks: Not "90% accuracy" (meaningless without context) but "precision 0.85, recall 0.80 on documents similar to yours"
• Disclosed failure modes: Where does the system systematically fail? ("Performs poorly on documents >50 pages" or "hallucinates citations 5% of the time")
• Evaluation data selection methodology: Was the data cherry-picked for demos or representative sampling of real-world cases?
• Model cards and red-team reports: Documentation showing training data sources, known biases, adversarial testing results

"We asked for evaluation data on our use case. Vendor A provided detailed metrics and failure analysis. Vendor B said 'trust us, it works great.' We went with Vendor A."

Dimension 3: Safety and Controls

Without safety controls, you're choosing between two bad options: check everything manually (productivity killer) or trust blindly (compliance disaster). Audit whether the system provides intelligent routing based on confidence and stakes.

Essential safety features:

• Confidence scores: Does the system provide a confidence score for each output? Can you trust that score (is it calibrated)?
• Configurable routing policies: Can you set rules like "escalate to human when confidence <0.7 OR stakes >$10k"?
• Guardrails: Are there content filters to prevent harmful outputs, policy violations, or regulated disclosures?
• Audit trails: Can you trace every decision the AI made, with full context, for compliance review?

Dimension 4: Model Swap-Out Policy

Models are perishable (Chapter 6). When GPT-6 or Claude Sonnet 5 or Llama 5 launches in 8 months, can you upgrade without renegotiating your contract? This isn't a hypothetical—it's a certainty.

What to verify:

• Architecture: Is the system hardcoded to a specific model (lock-in trap) or does it use swappable adapters (freedom to upgrade)?
• Decision control: Who controls the upgrade decision—you or the vendor? If the vendor, do they upgrade automatically (breaking your workflows) or never (leaving you stuck on old models)?
• Migration path: What happens if the vendor's preferred model becomes obsolete or expensive? Can you bring your own model?
• Cost implications: If newer models are cheaper, do you automatically benefit or is pricing locked?

A vendor response of "We use GPT-5" without an adapter layer is a red flag. Models are perishable; your architecture shouldn't be.

Dimension 5: Observability

Can you inspect prompts, inputs, outputs, costs, and drift in real time? Or is the system a black box where you deploy and pray?

Observability requirements:

If you can't see what's happening inside the system, you can't diagnose failures, optimise costs, or prove compliance. Black-box AI is a procurement non-starter.

Dimension 6: Ownership and Exit Rights

What happens if you need to leave? Who owns the data, the fine-tuning, the prompts, the evaluation harnesses? Can you take your work with you or are you starting from scratch?

Ownership audit:

• Input/output data: Do you own all data that passes through the system? Can you export it in standard formats?
• Fine-tuning and model weights: If you've fine-tuned models on your data, do you own those weights?
• Prompts and orchestration logic: Are prompts stored in your git repo or locked in the vendor's platform?
• Evaluation data and harnesses: Can you take your test sets and evaluation code with you?
• Exit timeline: How long does migration take if you leave? 30 days or 12 months?

If the vendor won't commit to data portability and reasonable exit timelines, they're planning on lock-in being their retention strategy.

The Capability Audit Checklist

Before signing a contract or committing engineering time, demand clear answers to these six questions:

Vendors who can't answer these questions clearly aren't selling AI capabilities—they're selling promises wrapped in proprietary lock-in. The ones who can answer are the ones worth evaluating.

"We turned the capability audit into our standard RFP. Half the vendors withdrew. The half that stayed could actually answer the questions. Made the decision easy."

"Check everything the AI does" kills productivity. "Trust everything the AI does" invites disaster. The answer is risk-based triage: route outputs intelligently based on confidence, stakes, and consequences. This is how you capture the upside without eating the downside.

The Impossible Choice

Without triage, organisations face two equally bad options:

• Option 1: Check everything manually. Review every AI output before it goes to production. This eliminates errors but also eliminates productivity gains. You've added an AI step to your workflow without removing human work—pure overhead.
• Option 2: Trust blindly. Auto-approve all AI outputs and hope for the best. This captures productivity gains until something goes catastrophically wrong—a compliance violation, a customer-facing error, a regulatory fine.

Both options lead to the outcomes we're seeing: 42% abandonment (organisations giving up after realising neither option works) and early financial losses from AI-related errors (EY's 2025 survey finding).

The Triage Framework: Three Dimensions

Not all AI outputs carry equal risk. Intelligent triage accounts for three factors:

Based on these three dimensions, you define routing policies that balance productivity against risk. High confidence + low stakes + reversible = auto-approve. Low confidence OR high stakes OR irreversible = require human review.

Defining Routing Policies

Routing policies map combinations of confidence, stakes, and reversibility to actions. Here's a practical framework:

Case Study: Customer Support Ticket Routing

Let's see triage in action for a concrete use case: AI-powered support ticket categorisation and response suggestions.

Without triage (the bad options):

• ❌ Check every AI suggestion → Agents spend same time reviewing as they would drafting responses → Zero productivity gain
• ❌ Trust all AI suggestions → Wrong categories, incorrect responses, customer complaints → Compliance nightmare

With triage (the smart approach):

With this triage setup, you auto-approve 60% of tickets (high confidence, low stakes), queue 30% for quick review (medium confidence/stakes), and escalate 10% for full human handling (low confidence or high stakes). The result: 60% × 100% productivity gain + 30% × 50% gain + 10% × 0% gain = ~60% overall productivity improvement, without the risk of blind trust.

"Before triage, we were checking everything. Zero productivity gain. After implementing confidence-based routing, we're auto-approving 55% of tickets with zero errors. That's where the ROI came from."

Implementing Triage in Practice

Here's how to operationalise risk-based triage in your organisation:

Step 1: Demand Confidence Scores from Vendors

If the AI system can't tell you how confident it is about each output, you can't triage. Confidence scoring should be non-negotiable in capability audits (Chapter 10). Verify that scores are calibrated—when the AI says 90% confidence, it should be right 90% of the time.

Step 2: Define Routing Policies Upfront

Don't deploy first and figure out triage later. Before production, stakeholders must agree on:

• Confidence thresholds for auto-approval (e.g., >0.9 for low-stakes, >0.95 for medium-stakes)
• Stake thresholds for mandatory review (based on dollar value, regulatory risk, customer impact)
• Escalation paths for edge cases (who reviews? What's the SLA?)
• Compliance overrides (categories that always require human review regardless of confidence)

Step 3: Log Everything for Auditing

Even auto-approved decisions should be logged with full context: input, AI output, confidence score, timestamp, user ID. If you later discover a systematic failure (e.g., AI was miscategorising a specific ticket type for three months), you need to identify which outputs were affected for remediation or customer notification.

Step 4: Tune Thresholds Based on Outcomes

Triage isn't "set and forget." As you gather production data, continuously tune:

• If false positives are high (AI says high confidence but is wrong), raise the auto-approve threshold
• If false negatives are high (AI says low confidence but is actually right), lower the escalation threshold
• If specific categories have worse performance, add category-specific routing rules
• As model accuracy improves (new model deployed), re-tune thresholds accordingly

Why Triage Is Non-Negotiable

Risk-based triage isn't paranoia or excessive caution. It's how you capture AI's productivity upside without eating its error downside. The math is straightforward:

• No triage, check everything: Productivity gain = 0%, error rate = 0%, organisation abandons AI as "not worth it"
• No triage, trust everything: Productivity gain = 100%, error rate = 5-15%, organisation suffers compliance violations and reputation damage, abandons AI as "too risky"
• With triage, intelligent routing: Productivity gain = 50-70% (auto-approve high confidence, quick review medium confidence), error rate < 1% (escalate low confidence and high stakes), organisation scales AI successfully

The 5% of organisations extracting consistent AI value aren't the ones with better models. They're the ones with triage systems that let them trust AI where it's reliable and escalate where it's not.

If the old playbook was "gather requirements → issue RFP → negotiate contract → implement → deploy," what's the new one? Here's how procurement transforms when you acknowledge that AI is perishable, probabilistic, and requires continuous experimentation.

The Old Playbook: Waterfall by Design

Traditional software procurement operated on long cycles because switching costs were genuinely high and implementations took quarters or years:

1. Requirements gathering: 3 months of stakeholder interviews, documentation, specification writing
2. RFP and vendor selection: 4 months of RFP issuance, vendor demos, evaluation, negotiation
3. Contract negotiation: 2 months of legal review, pricing discussions, SLA definition
4. Implementation: 6-18 months of deployment, customization, integration, testing
5. Go-live: Single cutover event, training, stabilization

Total timeline: 15-27 months from kickoff to production. This made sense when software was stable, requirements were knowable, and the technology you specified in month 1 would still be relevant in month 24.

The New Playbook: Procurement Becomes Continuous

AI procurement needs to operate on quarterly cycles, not multi-year contracts. Here's what changes:

Gates Replace Milestones

Traditional projects have milestones: requirements complete, design approved, development finished, UAT passed, go-live. AI projects need gates—decision points where you kill or continue based on empirical results.

The difference is critical. Milestones measure progress toward deployment. Gates measure evidence of value. You don't advance past a gate just because you completed the work—you advance only if the data says it's working.

This gated funding model prevents the "sunk cost trap" where organisations keep investing in failing AI because "we've already spent so much." Each gate forces a binary decision: does the data support continuing, yes or no?

"We used to measure 'percent complete.' Now we measure 'evidence of value.' If you can't show ROI data, you don't get past the gate."

Contracts Support Experimentation

Traditional contracts specify deliverables and timelines. AI contracts need to specify evaluation criteria and exit ramps.

What new AI contracts look like:

• Proof-of-value period: 30–90 day trial with clear success metrics written into the contract. Either party can exit with no penalty if metrics aren't hit.
• Model swap rights: Customer can switch underlying models without vendor approval or additional fees. Contract binds platform access, not specific model versions.
• Data ownership: Customer owns all inputs, outputs, fine-tuning data, and evaluation harnesses. No vendor lock-in through proprietary data formats.
• Observability guarantees: Full access to prompts, logs, costs, performance metrics in exportable formats. Vendor can't hide black-box behavior behind proprietary interfaces.
• Performance SLAs: Not uptime (easy to game), but accuracy/precision/recall on customer's evaluation data. Vendor commits to maintaining performance thresholds or customer can exit.

Funding Follows Value, Not Projects

Traditional IT budgeting funds projects: "We're allocating $2M for the CRM upgrade." AI budgeting should fund capabilities and release funds in tranches as value is proven.

Staged funding model:

• Seed funding ($50k): Offline eval and shadow deployment. Low investment, fast learning. If this fails, you've killed it for $50k instead of $2M.
• Tranche 2 ($200k): Released only if Gate 2 passes (shadow deploy succeeds, real-world failure modes understood, path to ROI clear).
• Tranche 3 ($500k): Released only if Gate 3 passes (bounded rollout hits target metrics, unit economics positive at scale).
• Full funding (scale investment): Released only when unit economics work at production volume. No faith-based funding.

This aligns spending with evidence. You don't fund the full implementation until you've proven the hypothesis with progressively higher fidelity testing.

Roles Change: From IT Project to Product Initiative

Traditional procurement is led by IT/ops teams executing stakeholder requirements. AI procurement needs a product manager with AI literacy, supported by cross-functional teams.

It's not an IT project with a go-live date. It's a product initiative with continuous experimentation, measurement, and performance management.

The Playbook in Practice: Timeline Comparison

Let's compare timelines for deploying AI-powered customer support using old vs new playbooks:

The new playbook delivers 75% faster time-to-value and 95% lower sunk costs if you need to kill the initiative.

The 42% abandonment rate isn't just a deployment problem—it's an economics problem. Organisations are scaling proofs-of-concept, not value. Here's how to structure AI economics so they actually deliver return on investment.

Unit Economics Over Project Budgets

Traditional IT projects measure success by "on time, on budget, on spec." The project either completes or it doesn't. AI projects need to measure unit economics: dollars saved or revenue generated per 1,000 AI requests, after all costs.

This shift matters because traditional projects succeed when they deploy. AI projects succeed when they deliver ongoing value at sustainable cost. Deployment is just the beginning.

Example: Customer Support Ticket Triage That Works

Let's work through a real scenario with actual numbers:

Current baseline:

• 1,000 tickets per day coming into support queue
• Average 15 minutes to read, categorise, and route each ticket
• Support agent cost: $40/hour fully loaded → $10 per ticket
• Total daily cost: $10,000 in labor

AI hypothesis:

• AI categorises tickets in seconds, suggests responses based on category
• Agents review and approve AI suggestions in 3 minutes instead of drafting from scratch in 15 minutes
• 80% of tickets handled with AI assist, 20% too complex and escalated to full manual handling

AI economics breakdown:

Net value calculation:

• Daily savings: $10,000 (baseline) - $4,200 (AI) = $5,800/day
• ROI: $5,800 / $4,200 = 138% daily return
• Annual savings: $5,800 × 365 = $2.1M/year
• Payback on $50k implementation: 9 days

Counter-Example: Contract Review That Fails Economics

Not every use case works financially. Here's why some AI deployments get killed despite technical success:

Current baseline:

• 50 contracts per month reviewed by legal team
• 2 hours per contract × $200/hour (attorney cost) = $400 per contract
• Total monthly cost: $20,000 in legal review time

AI hypothesis:

• AI reads contracts, flags risky clauses, suggests redlines based on playbook
• Legal reviews AI output and makes final judgment calls
• Expectation: Save 30-40% of attorney time per contract

AI economics (the disappointing reality):

Net value (underwhelming):

• Monthly savings: $20,000 - $15,600 = $4,400/month
• ROI: $4,400 / $15,600 = 28% return
• But: Only saves 30 minutes per contract, and attorneys report lower confidence in AI-flagged issues
• Reality: Time "saved" is spent verifying AI didn't miss critical clauses

Avoiding the "Scaled Proof" Trap

Gartner and S&P's failure statistics are full of organisations that scaled proofs-of-concept instead of scaling proven value. Here's how it happens:

1. POC works on 100 test cases → Impressive demo, stakeholders excited
2. Exec sponsors greenlight full deployment → Budget approved, team mobilized
3. Production rollout reveals edge cases → Accuracy degrades on real-world diversity
4. Hidden costs emerge → More human review needed than predicted, infrastructure costs higher than expected
5. Unit economics flip negative at scale → What worked for 100 cases loses money at 10,000
6. Project gets quietly shelved → Another contribution to the 42% abandonment rate

"The POC was perfect. But perfect on 100 cherry-picked examples. At 10,000 production cases, the accuracy dropped 20 points and costs tripled. We scaled the proof, not the value."

To avoid this trap: measure unit economics at every gate (Chapter 12). Don't scale until you've proven positive ROI in bounded production at realistic volume, not just POC demos.

The Time-to-First-Value Metric

Traditional IT measures "time to deployment." AI should measure time to first dollar of value—how quickly you generate measurable savings or revenue that exceeds costs.

What to track:

• Time to hypothesis validation: How many days from "we think AI can help with X" to "offline eval confirms it works"? Target: < 14 days.
• Time to bounded production: How many days from validated hypothesis to live production with 10% of traffic? Target: < 60 days.
• Time to positive cumulative value: How many days until cumulative savings exceed all costs including implementation? Target: < 90 days.

If time-to-first-value exceeds 90 days, you're probably building too much infrastructure or chasing the wrong use case. The winners are shipping measurable value in weeks, not quarters.

Making the Economics Work: Checklist

Before deploying any AI initiative, verify unit economics pass these tests:

If you can't check all six boxes, don't scale. Either redesign the hypothesis, reduce scope, or kill it and move to a use case with better economics.

Here's the uncomfortable truth that most AI vendors won't tell you: the AI isn't the hard part. The hard part is changing how people work. BCG's study found that the 5% of companies extracting consistent AI value invest heavily in change management, workflow redesign, and new roles. The organisations failing are the ones that deploy AI and expect people to just... use it.

The Deployment Fallacy

Most AI deployments fail this basic test: they automate the current process instead of redesigning the workflow around AI capabilities. This is automation theater—adding steps without removing work.

Create New Roles, Don't Just Augment Old Ones

AI doesn't just augment existing roles—it creates entirely new ones that didn't exist before. The organisations winning are the ones creating these roles explicitly, with clear responsibilities and career paths.

"We tried making AI everyone's job. It became no one's job. Once we created dedicated roles, adoption went from 15% to 80% in three months."

Train AI Literacy, Not Tool Features

Traditional software training is "here's how to use the CRM: click here, enter data here, hit save." AI training needs to be fundamentally different: "here's how AI works, here's where it fails, here's when to trust it and when to override it."

Key concepts every AI user needs to understand:

• Probabilistic outputs: AI gives you a probably-correct answer, not a definitely-correct one. Confidence scores matter. A 0.95 confidence prediction is different from 0.70—learn to act on that signal.
• Systematic failure modes: Where does this specific AI systematically fail? (e.g., "performs poorly on non-English names" or "hallucinates citations when confidence <0.75"). Don't discover this through customer-facing errors.
• Escalation criteria: When should you ignore the AI suggestion and do it manually? When should you escalate to a specialist? This needs to be taught explicitly, not learned through trial and error.
• Feedback loops: How do you report failures so the system improves? If users see errors but don't report them, the system can't learn. Make feedback mechanisms explicit and easy.

Measure Adoption, Not Deployment

Deployment is "we turned it on." Adoption is "people are actually using it and getting value." The gap between the two explains most AI failures.

Adoption metrics that matter:

• Active usage rate: What percentage of eligible users are actively using the AI? Not "logged in once" but "use it daily for >50% of eligible tasks"
• AI suggestion acceptance rate: What percentage of AI outputs are accepted vs manually overridden? Low acceptance = trust problem or accuracy problem
• Workaround detection: Are users routing around the AI? (e.g., copy-pasting AI outputs into manual tools, using shadow spreadsheets, reverting to old workflows)
• Net Promoter Score from users: Would frontline workers recommend this AI to colleagues? NPS < 0 = adoption will collapse
• Time-to-proficiency: How long does it take new users to become productive? If it takes weeks, you have a training or UX problem

Iterate Based on User Feedback

AI isn't "ship and forget." It's "ship, observe, tune, repeat." BCG's "future-built" firms treat AI as a product with a continuous improvement cycle.

The feedback loop in practice:

1. Deploy to 10% of users (early adopters who'll provide detailed feedback)
2. Collect feedback systematically on where AI fails, where it surprises positively, where workflows need redesign
3. Tune based on data: adjust prompts, refine triage thresholds, retrain on newly discovered edge cases
4. Expand to 30% of users, incorporating lessons from the first 10%
5. Repeat the cycle until you reach 80-90% adoption with sustained positive NPS

The organisations failing are the ones that deploy to 100% on day one, then wonder why adoption is low and errors are high. Gradual rollout with continuous tuning is how you get to durable ROI.

"We shipped to 100% of agents on launch day. Adoption was 12%. We rolled back, redeployed to 10%, fixed issues, expanded gradually. Six months later we're at 85% adoption."

The Difference Between Demo Magic and Durable ROI

Demo magic is easy: controlled environment, cherry-picked examples, no edge cases, no organisational friction. Everyone leaves the demo room excited. Then reality hits.

Durable ROI requires:

• Workflows redesigned around AI strengths, not AI bolted onto broken processes
• New roles to manage and tune the systems—AI Product Owner, Prompt Evaluator, Triage Analyst, MLOps
• Training on AI literacy so people understand when to trust, when to override, how to provide feedback
• Gradual rollout with feedback loops to improve performance based on real-world usage
• Adoption measurement that tracks actual usage, not deployment completion

This is change management as product work. It's messy, iterative, and requires continuous investment. But it's also the difference between the 5% who extract consistent AI value and the 42% who abandon their initiatives.

BCG's research on the 5% of companies extracting consistent AI value reveals a pattern. These organisations aren't smarter, luckier, or better funded. They're doing fundamentally different things—things that look obvious in hindsight but require discipline to execute.

They Embed AI Into Core Workflows, Not Staple It On

Losers treat AI as a feature addition: "We added an AI chatbot to the website." "We enabled AI-suggested replies in the CRM." The AI sits adjacent to work, not integrated into it.

Winners redesign the workflow from first principles: "We rebuilt our customer service workflow so AI handles tier-1 queries end-to-end, agents focus on tier-2+ complexity, and escalations route automatically based on confidence scores."

The difference is integration depth. Stapled-on AI is optional—people can ignore it, route around it, or begrudgingly use it. Embedded AI is part of the standard operating procedure. It's not "would you like AI help?" but "this is how we work now."

They Start With High-Volume, Low-Stakes Use Cases

Losers chase the sexy use cases first: "AI-powered strategic planning." "Automated financial forecasting." "AI-driven M&A analysis." High stakes, complex workflows, low error tolerance—exactly where AI is weakest.

Winners start boring: "AI categorises support tickets." "AI extracts invoice data." "AI summarises meeting notes." High volume, low stakes, clear success criteria, easy to measure improvement.

Why boring wins:

• Build evaluation harnesses on abundant data: You have 10,000 historical support tickets, not 50 strategic plans. More data = better testing.
• Iterate quickly without regulatory risk: If ticket categorisation fails, you fix it. If financial forecasting fails, regulators investigate.
• Prove ROI in weeks, not quarters: High-volume use cases deliver measurable value fast. Strategic use cases take months to validate.
• Build organisational AI literacy on forgiving problems: Learn on low-stakes tasks where mistakes are recoverable, not high-stakes decisions where errors are catastrophic.

Once you've proven the model works on boring problems and the organisation understands how to deploy AI successfully, then you tackle strategic ones. Boring first. Sexy later.

"We wanted to start with AI-driven pricing strategy. Our consultant said 'start with invoice processing.' We rolled our eyes. That boring use case paid for itself in 6 weeks and taught us how AI actually works."

They Treat AI as a Capability to Continuously Improve, Not a Project to Complete

Losers think in projects: "We're implementing the AI solution. Go-live is Q3. Then we're done." Project managers track completion percentage. Success is measured by hitting the go-live date.

Winners think in capabilities: "We're building an AI-augmented support capability. Initial deployment is Q3. Then we tune prompts, retrain on failures, adjust triage thresholds, swap models quarterly, and measure ROI continuously."

The project mindset leads to "deploy and forget." The capability mindset leads to continuous improvement, which is where durable ROI comes from. AI isn't software you install—it's a capability you cultivate.

They Instrument Relentlessly

Losers deploy AI and hope it works. Winners deploy AI and measure whether it works with obsessive detail.

What winners instrument:

• Log every input, output, confidence score, user override: Full audit trail for debugging and compliance
• Track unit economics per workflow: Cost per request, time saved per transaction, error rate, value delivered
• Monitor drift continuously: Is performance degrading as input patterns change? Are new edge cases appearing?
• Run A/B tests on models: GPT-5 vs Claude Sonnet 4.5 vs Llama 3.3 on the same tasks, pick the winner based on data
• Publish internal dashboards: Everyone sees what's working and what's not—transparency drives accountability

Instrumentation isn't bureaucracy. It's how you know when to scale (ROI positive, adoption high) and when to kill (economics don't work, accuracy degrading). Without measurement, you're flying blind.

They Build for Model Swapping from Day One

Losers hardcode dependencies: "Our system uses GPT-5." When GPT-6 launches or Claude Sonnet 5 proves superior, they're stuck renegotiating contracts or rewriting code.

Winners abstract models from the start: "Our system uses a swappable LLM adapter. We're currently routing to GPT-5, but we can switch to Claude Sonnet 4.5 or Llama 3.3 with a config change and retest against our evaluation harness."

This isn't over-engineering or premature optimization. It's acknowledging that models are perishable (Chapter 6) and building for the inevitable. Model improvements happen quarterly. Your architecture should support quarterly upgrades.

They Invest in AI Literacy Across the Organisation

Losers treat AI as an IT initiative led by engineering. Winners treat it as an organisational capability requiring cross-functional fluency.

They Kill Failures Fast

Losers keep funding underperforming AI projects because "we've already invested so much." Sunk cost fallacy at organisational scale. Projects limp along for quarters, consuming budget and management attention, delivering no value.

Winners gate funding and kill ruthlessly based on data:

• Hypothesis fails offline eval? Kill it immediately, document why, move to next use case. Cost: $5k, 2 weeks elapsed.
• Shadow deployment reveals systematic failures? Kill it, try different approach or different model. Cost: $25k, 6 weeks elapsed.
• Bounded rollout hits targets but unit economics don't work at scale? Kill it before full deployment. Don't scale loss-makers. Cost: $75k, 12 weeks elapsed.

The 42% abandonment rate includes both smart kills (failed experiments stopped early based on data) and expensive failures (projects that scaled before proving value). Winners optimize for fast, cheap kills based on empirical evidence, not executive opinion.

"We've killed seven AI initiatives in the past 18 months. The three we kept generated $4M in value. Killing fast is how we found the winners."

The Pattern: Different Actions, Not Different Resources

Here's what's striking about BCG's 5%: they didn't buy better AI. They didn't have bigger budgets. They didn't hire armies of PhDs. They bought differently—with experimentation, measurement, and continuous improvement baked in. And they built differently—for perishability, composability, and organisational learning.

The gap between 5% success and 42% abandonment isn't about AI technology. It's about procurement models, architectural choices, and operating discipline. The playbook is available. The tools exist. The difference is execution.

Reading about AI procurement transformation is one thing. Doing something about it is another. Here's what to do in the next 24 hours—specific, actionable steps based on where you are right now.

If You're Currently Evaluating AI Vendors

Stop the traditional RFP process. Seriously. If you're running a requirements-checklist procurement with six-month evaluation timelines, you're optimizing for the wrong things and setting yourself up for the 42% abandonment rate.

"We paused our 4-month RFP, ran a 10-day test on actual data with three vendors. Had our answer in two weeks. Saved us four months and $200k in evaluation costs."

If You've Already Deployed AI and It's Underperforming

Instrument before you optimize. You can't fix what you can't measure. Most underperforming AI deployments lack basic observability, making it impossible to diagnose why they're not delivering value.

If You're Planning an AI Initiative

Start with architecture, not use cases. If you build for one use case and hardcode dependencies, you'll have to rebuild for the next one. Architecture designed for perishability and swappability pays dividends across all future initiatives.

If You're a CTO or Procurement Leader

Change how you fund and gate AI projects. Stop running them like traditional IT implementations with fixed scopes and go-live dates. Shift to hypothesis-driven experimentation with staged funding.

If You're Skeptical AI Will Ever Work

You might be right—but test it before deciding. Your skepticism could be valid (wrong use case for your organisation) or it could be based on bad vendor experiences. The only way to know is empirical testing.

The One Thing You Must Do

If you only do one thing from this entire book, do this:

The organisations winning aren't the ones with better vendors. They're the ones with better procurement and deployment models. They're hypothesis-driven, not requirements-driven. They're measuring unit economics, not checking boxes. They're architecting for perishability, not pretending models are durable. They're building AI literacy across the organisation, not treating it as an IT project.

None of this requires bleeding-edge technology or massive budgets. It requires discipline to do procurement differently when the technology is fundamentally different.

The Shift Starts Today

The 42% abandonment rate isn't inevitable. The 5% who succeed aren't lucky or smarter. They're doing procurement, architecture, and operations differently because the technology demands it.

Every chapter in this book pointed to the same truth: AI is perishable, probabilistic, and requires continuous experimentation. Traditional software procurement—requirements documents, multi-year contracts, feature checklists, one-time deployments—is catastrophically mismatched to that reality.

You can't RFP your way to AI maturity. You can't install-and-forget. You can't treat probabilistic capabilities like deterministic features. You can't lock into three-year contracts when models improve quarterly.

But you can do procurement that acknowledges perishability. You can build architecture that supports swappability. You can measure unit economics and kill failures fast. You can train AI literacy and redesign workflows. You can treat AI as a capability to cultivate, not a product to install.

The playbook exists. The tools are available. The question is: will you use them, or will you become part of the 42%?

The shift starts today. What will you do in the next 24 hours?