How Many Walls Stand Between PoC and Production?

The general who wins a battle makes many calculations in his temple before the battle is fought. The general who loses a battle makes but few calculations beforehand. — Sun Tzu, The Art of War, "Laying Plans"

"The PoC ran in a week — why does going to production still take another six months?" Every customer asks this sooner or later, and most FDEs fumble the answer.

The answer isn't actually hard. What holds up a launch was never the amount of code.

I. Between PoC and Production Stand a Lot of Walls

Start by taking apart the word "productionize." Different people mean completely different things by it — one means "add monitoring," another means "add more machines," a third means "pass the security audit." None of them is wrong. Each is just a piece.

A system that can actually do work inside a real enterprise splits into three layers: the model, the interface, and the harness. The model can reason, but it can't act — the capacity to act comes entirely from the harness. In one insurance company's deployment, the harness comes down to five things: tools (the model can't calculate premiums reliably, so give it a calculator; it can't read a handwritten form, so give it a recognition router); integration (wiring into each insurer's wildly different API — and where there's no API, driving the browser programmatically instead); a file system (a process that runs for days needs its state stored somewhere, so an interruption doesn't mean starting over); task orchestration (route the easy classification work to a cheap model, the ambiguous clauses to a stronger one, with a task graph deciding what runs when); and a supervision mechanism (low-confidence outputs go to a human review queue, irreversible actions wait for human approval, and everything leaves an audit trail)¹.

The value in this split is honesty: a company on the front lines gave a name to "everything that sits between the model and production," and admitted it's half the product, not some cleanup task tacked on at the end. An open-source book on enterprise agent platform engineering breaks it down further still — it lists a dozen-plus capabilities a production-grade deployment has to deliver, from the runtime and tool contracts to evaluation and cost governance, each paired with a minimal code reference².

This book follows the same direction, breaking down what stands between a model and production into eight walls — translating "productionize" into a checklist you can sign off item by item. All eight walls have to come down before delivery is complete.

II. The Eight Walls

Wall One, data and permissions. The question on the ground: which tables can be read, who can see the results, how does masking work, what's the process for exceptions? The customer's security review isn't a formality — there's an actual form to fill out. What gets you over the wall: a Data Contract. That's the subject of Chapter 10.

Wall Two, integration. The question on the ground: the system the agent needs to write back to is one the customer has been running for twenty years — no API, and nobody can quite describe the data format anymore. Box's CEO put it bluntly: integration is the wall, and an agent won't fix your integration problems for you — it'll just run straight into them³. What gets you over the wall: a path that actually connects the old system to the new.

Wall Three, tool boundaries. The question on the ground: which tools can the agent call, and who validates the parameters? An agent that can send an email on its own and one that can only draft an email sit at two completely different risk levels. What gets you over the wall: a Tool Contract. That's the subject of Chapter 11.

Wall Four, controllable execution. The question on the ground: if a job runs wild at 2 a.m., whose fault is it? If it's stuck waiting on human approval, can it pick back up the next day? What gets you over the wall: a Run State Machine with checkpoints. Also covered in Chapter 11.

Wall Five, provable results. The question on the ground: after a prompt change, did quality go up or down? If you can't answer that, every change is a gamble. What gets you over the wall: an eval set and a regression gate. That's the subject of Chapter 12.

Wall Six, trustworthy results. The question on the ground: the customer points at the screen and asks "where did this number come from, why should I believe this conclusion" — what can you produce besides an apology and a rerun? What gets you over the wall: a chain of evidence. That's the subject of Chapter 13.

Wall Seven, cost and stability. The question on the ground: a call cost pennies in the demo, and the monthly bill hits five figures after launch; the system worked fine yesterday and is glitching today. What gets you over the wall: cost attribution and a stability commitment. That's the subject of Chapter 14.

Wall Eight, transferable responsibility. The question on the ground: who's responsible when something goes wrong, and once the FDE leaves, who does the system get handed to? What gets you over the wall: a responsibility handoff table and a runbook — Chapter 8 already sketched an early version of that table; Chapter 17 unpacks the full handoff.

A concrete example. Jove Zhong, who heads AI Agent FDE at Cresta, has described just how complex a voice AI agent can get: as many as twenty models can be running behind the scenes at once — ASR, interruption detection, noise isolation, retrieval, tool calls, guardrails spanning multiple concurrent models — on top of compliance audits (PCI for payment cards, HIPAA for U.S. medical privacy) that routinely take six months to a year. Getting it running end to end takes a week; writing the thousands, sometimes tens of thousands, of tests takes a month after that — and these aren't traditional unit tests, but small models trained on historical call data to actively simulate the edge cases a real deployment will hit (more on this technique in Chapter 12)⁴.

This example puts a concrete face on both Wall Four (controllable execution — which of twenty models runs when, which one gets to interrupt which) and Wall Five (provable results — where thousands of tests come from) at the same time: the excitement of "end to end in a week" during the PoC stage and the reality of "a month of testing" afterward are two sides of the same project.

Looked at together, the eight walls aren't all the same kind of thing: Walls One and Two are the foundation — without them, nothing else is worth discussing. Walls Three and Four are risk — without them, the system can still run, but something can go wrong at any moment. Walls Five through Eight are trust — without them, the system stays stuck at "trial" forever. Different natures call for different fixes, and different acceptance criteria (see Figure 9-1).

Figure 9-1: Model capability is only the starting point — a production system still has to pass through eight kinds of engineering and organizational constraints.

III. Map the Walls When You Sign the Contract — Not After You've Already Hit Them

Why map the walls out? Because the engineering effort behind each one has to go into the quote and the schedule.

Two different approaches to the contract lead to two completely different endings.

In a contract that has mapped the walls, every wall comes with an effort estimate, an owner, and acceptance criteria — behind the quote the customer sees is a map, both sides move forward by milestone, and hitting a wall just means it was on the plan all along.

In a contract that hasn't, "AI capability delivery" gets bundled into one line item — the applause from the demo is still ringing, and delivery starts running into walls one after another. Every wall becomes "an additional requirement," every wall means renegotiating money and schedule from scratch, and three months of wrangling later the project quietly dies. Same project — call the first outcome a delivery and the second a write-off. The fork in the road is only whether the walls got mapped on the day the quote was written.

There's another cost to not mapping the walls, and it lands on the vendor's own books.

A delivery lead with hands-on industry experience has warned about this: take a $2 million contract that hasn't priced in the engineering effort behind the walls, staff it with six engineers for twelve months, and the gross margin can come out negative — profitable on paper, eroding enterprise value in reality⁵. The full arithmetic behind that belongs to Part Five; for now, here's the warning worth flagging: failing to map the walls is a direct cause of the losses that follow.

The popular claim that "AI projects have a high failure rate" sounds like a verdict on the technology. Take "failure" apart and most stalled projects die of the same cause: at quoting time, the walls never made it into the contract, and never made it into the price. A wall that never got priced doesn't stop existing — it just surfaces later, at the worst possible moment: mid-delivery, under the customer's gaze. The technology doesn't deserve the blame.

IV. As Models Get Stronger, Do the Walls Disappear on Their Own?

A natural question: as each model generation gets stronger, will these eight walls just disappear?

Half of them shrink. The first four walls — data access, integration, tool generation, run orchestration — are engineering work that code can solve, and the stronger the model, the faster that work gets done. Some have gone even further and turned FDE work itself into a product: an agent that takes over the repetitive labor behind the first four walls⁶ — pointed in the same direction, just packaged differently.

But the last four walls are hard to shrink.

Ali Ghodsi, Databricks' co-founder and CEO, gave an example in a guest lecture at Stanford that draws this line clearly: building a production-ready application connector used to take about nine months on their old process — the first quarter alone went to requirements gathering and dozens of pages of documentation, and only then came development, staging, security verification, and customer feedback. The team lead wrote a working version himself with AI in two days, but the team agreed it was still just a demo — it hadn't gone through full testing and couldn't be guaranteed safe and stable for a customer. The more sobering estimate: even bolting AI straight onto the old process only shrank the cycle from nine months to about seven and a half — the time saved got eaten right back up by the process itself. The real turning point wasn't a stronger model. It was rewriting the process: requirements went from a full quarter down to a week, staging environments were handed to a team better suited to building them in parallel, and the way people worked changed from "one person owns one connector" to "a team collectively covers a group of connectors." After the rebuild, they shipped seven production-grade connectors in a single quarter⁷.

The AI that wrote a demo in two days ate into the engineering effort behind the first four walls. Nine months becoming seven and a half — when only the model changes and the process doesn't — shows that the last four walls won't give an inch. Seven connectors in one quarter shows that once the last four walls come down, the speedup from the first four finally converts into productivity. Testing, security, how people collaborate, how responsibility is divided — the height of these walls is set by organization and trust, not by model capability. This is the engineering version of the judgment from Chapter 5: the reason an FDE won't be replaced by AI is hiding inside these last four walls.

V. Take This Table With You

This chapter doesn't hand you the specific method for taking down each wall — that's the job of the next five chapters. What it hands you is a reference table: take any project that's "passed the PoC, awaiting production" and ask three questions about it — has this wall come down? What's the deliverable? Who signs off?

Wall	The question on the ground	What gets you over it	Full treatment
One — Data and permissions	Which tables can be read, who can see the results	Data Contract	Chapter 10
Two — Integration	How a new system writes back to a twenty-year-old one	A connected integration path	Chapter 10
Three — Tool boundaries	What the agent can call, who validates the parameters	Tool Contract	Chapter 11
Four — Controllable execution	Whose fault if it runs wild, can it resume once stuck	Run State Machine and checkpoints	Chapter 11
Five — Provable results	Did a prompt change make it better or worse	Eval set and regression gate	Chapter 12
Six — Trustworthy results	What answers "where did this number come from"	Chain of evidence	Chapter 13
Seven — Cost and stability	What to do about a runaway bill or a jittery system	Cost attribution and a stability commitment	Chapter 14
Eight — Transferable responsibility	Who's responsible when it breaks, who inherits it after the FDE leaves	Responsibility handoff table and runbook	Chapter 17

If any one of the three questions goes unanswered, that wall is a hole in the schedule — and the most expensive kind of hole. The full version of this table is Appendix A: the delivery checklist, organized across four stages — discovery, minimum viable deployment, productionization, and launch and adoption. This chapter's eight walls sit concentrated in the "productionization" stretch, grouped into foundation, risk, and trust.

Starting next chapter, the walls come down one by one. First, the one you hit earliest: the customer's data and permissions.

---

Footnotes

1. "Inside an Applied AI Company" (Pace long-form post) ↩

2. enterprise_agent_platform (open-source enterprise agent platform engineering project) ↩

3. a16z, Box CEO on AI agents and why enterprises can't keep up ↩

4. Tencent Research Institute, AI Lens Roundtable, Episode 6: "FDE Non-Consensus Views and a Field Guide from Silicon Valley Founders" ↩

5. @deployengineer (Bhaulik Patel), essay series ↩

6. Palantir Foundry, "AI FDE" product documentation ↩

7. Ali Ghodsi, guest lecture, Stanford MS&E 435 ↩