FDE Delivery Checklists

How to Use This Checklist

• Each item is marked: ✅ met / ⚠️ partial / ❌ not met / N/A not applicable
• The rule: if any phase's critical item (marked ★) is unmet, do not move on to the next phase
• Consequence notes: what typically happens when an item goes unmet (red-line risks)
• Chapter references: each item lists the chapter it comes from, for looking up the details

---

Phase I: Discovery (Chapter 6)

Phase goal: locate a high-frequency, high-value, verifiable problem inside the customer's workflow; define the baseline and the success metric first.

Critical items (★)

• ★ The problem is clearly defined

  • Elements: a one-sentence problem statement + current baseline data + a target metric
  • See: Chapter 6
  • If unmet:
    • A vague problem definition → acceptance never reconciles with the numbers later
    • No baseline data → no way to prove "better than before" after launch
    • No target → the project stays forever at "pretty good, let's try a bit more"
  • Checklist: [ ] defined clearly in one sentence [ ] baseline numbers have a source [ ] target value and time window are frozen

• ★ The baseline has been measured

  • Elements: hard numbers (not estimates) for the manual process's time / error rate / cost
  • See: Chapter 6
  • If unmet:
    • Only qualitative assessments ("slow", "error-prone") → no quantified before/after comparison
    • Test sample too small or unrepresentative → the data isn't credible and the customer won't accept it
    • Measurement method not kept consistent → before and after get measured on different scales, and the numbers explode
  • Checklist: [ ] sample size ≥ 30 [ ] measurement method documented [ ] measured at least twice

• ★ Users have tried an interactive prototype and given real feedback

  • Elements: one end-to-end interaction on real data with real users, collecting genuine feedback (not just "they nodded")
  • See: Chapters 6 and 7
  • If unmet:
    • The prototype only ever ran on demo data → at launch, the data's distribution and format turn out completely different
    • Users nodded but never really used it → the understanding of the requirement never got past the customer's one sentence
    • No real feedback collected (how customers use it, what they gripe about) → version two is guesswork again
  • Checklist: [ ] real data used (real distribution, real volume) [ ] at least 1 real front-line user has tried it [ ] user feedback / questions / points of confusion recorded

Non-critical items (optional but recommended)

• Data-reachability checklist completed

  • Contents: where the data is / who owns it / who can see it / masking rules / the exception process
  • See: Chapter 10
  • Why do it early: missing data, missing permissions, and legal problems get caught while there's still time to fix them

• Tolerance for change confirmed

  • Contents: the process owner knows, the affected people are identified, and the blast radius of the change has been calculated
  • See: Chapter 15
  • Why do it early: avoids "a flawless launch and broken users"

---

Phase II: Minimum Viable Deployment, MVD (Chapters 7 and 12)

Phase goal: use real data and real users to test quickly whether the solution actually works; use the smallest possible scope to keep the cost of acceptance under control.

Critical items (★)

• ★ Real data (real distribution, not demo data)

  • Elements: volume, distribution, and format all match the production environment; not "we picked our best 100 records"
  • See: Chapter 7
  • If unmet:
    • Demo data with a simple distribution → 95% in the PoC, 65% in production, and a customer complaint about "shrinkage"
    • Demo data volume too small → second-level responses in the PoC, hundred-scale latency at launch
    • Demo never covered edge cases → one class of data goes in and the system falls over
  • Checklist: [ ] data volume representative of the production environment [ ] edge cases at least 5% of the set [ ] sources traceable

• ★ Real users (at least one front-line user genuinely using it every week)

  • Elements: not "the boss took a look" — the people who actually do the work, using it weekly, able to give feedback
  • See: Chapter 7
  • If unmet:
    • Only management signed off → the people who execute don't cooperate, and the feature design drifts completely away from the use case
    • The users aren't the people doing the work → the workflow's real bottlenecks never surface
    • The users don't use it often → sporadic use never exposes stability problems
  • Checklist: [ ] user list has names / departments [ ] users are the actual operators (not the buyers) [ ] weekly usage records exist

• ★ Golden Set prepared (at least 50 representative examples, with a designated judge)

  • Elements: collect 50 examples representing real front-line scenarios ("this is how the problem should be solved"); explicitly designate one person responsible for judging correctness against them
  • See: Chapter 12
  • If unmet:
    • Too few samples (<20) → the real diversity isn't represented and the evaluation results are void
    • No designated judge → labeling disputes can't be resolved and the work gets redone
    • Biased samples → only the scenarios the customer cares most about, missing 80% of the actual problems
  • Checklist: [ ] at least 50 samples [ ] samples cover at least 3 major scenarios [ ] one judge designated [ ] version history kept for the samples and the final eval set

Non-critical items (optional)

• Scope boundaries made explicit
  • Contents: draw the boundary clearly — one process, one user group, and the agent takes only the measurable segment
  • See: Chapter 7
  • Recommended early: keeps the MVD from becoming "let's try every feature"

---

Phase III: Production (Chapters 8–14)

Phase goal: build the PoC out into a system that runs on its own in a real enterprise environment. Take down the "eight walls" from Chapter 9 one at a time.

Critical items (★) — the walls that must come down

Foundation (if these don't go through, nothing else is worth discussing)

• ★ Wall One: data and permissions → sign the Data Contract

  • Contents: scope / visibility / retention / audit — four items on one sheet
  • Elements: which tables can be read, who can see the results, how masking works, what process handles exception permissions
  • See: Chapters 9 and 10
  • If unmet:
    • Only after launch does it turn out a table can't be read → the process jams
    • No masking agreement → the customer complains about sensitive information leaking
    • Result visibility never defined → some stakeholders can't see the output, and the project fails
  • Checklist: [ ] all data sources to be read listed on one sheet [ ] permission owner marked for each source [ ] masking rules attached as a table [ ] audit-log plan confirmed

• ★ Wall Two: integration → an integration path that actually connects the old system to the new

  • Contents: how to connect to the system the agent writes back to (API / database / RPA / other)
  • Elements: interface protocol confirmed, data-format mapping, retry-on-failure policy, idempotency guarantees
  • See: Chapters 9 and 10
  • If unmet:
    • API availability never verified up front → after launch, the interface turns out never to have actually worked
    • No retry on failure → a transient network blip loses data
    • No idempotency check → the retry processes the customer's data twice
  • Checklist: [ ] integration approach PoC'd (connected for real at least once) [ ] network errors, timeouts, and malformed responses all tested [ ] idempotency design documented [ ] rollback-on-failure plan explicit

Risk (it runs, but something can go wrong at any moment)

• ★ Wall Three: tool boundaries → a Tool Contract

  • Contents: which tools the agent can call, who validates the parameters, when a call gets refused
  • Elements: tool inventory / parameter-validation rules / safeguards on high-risk tools (sending email, modifying data)
  • See: Chapters 9 and 11
  • If unmet:
    • No parameter validation → the agent sends the wrong email or deletes the wrong record
    • No tool whitelist → the agent calls tools it should have no access to
    • No rate limits → the agent fires DDoS-level request volumes at an external API
  • Checklist: [ ] tool inventory whitelisted [ ] parameter-validation rules for every tool written into code [ ] approval tickets required for high-risk tools (write operations, outbound sends) [ ] call-rate and concurrency limits confirmed

• ★ Wall Four: controllable execution → a Run State Machine + Checkpoint & Resume

  • Contents: what happens when a task fails, whether a run stuck on human approval can resume, whether a mid-run crash can pick up where it left off
  • Elements: state-transition diagram / checkpoints / recovery policy
  • See: Chapters 9 and 11
  • If unmet:
    • No checkpoints → a failed task reruns from scratch and customer data gets processed twice
    • No suspension point for human approval → anything needing human confirmation jams and blocks the whole chain
    • No degradation path → a system failure is a business failure, with no manual fallback
  • Checklist: [ ] state-transition diagram drawn [ ] at least 3 critical checkpoints implemented [ ] the human-approval node can suspend and resume [ ] the fallback-to-manual/Excel path rehearsed

Trust (without these, the system stays stuck at trial forever)

• ★ Wall Five: provable results → an eval set + a release gate for changes

  • Contents: after a prompt / model / tool change, the data says whether results went up or down
  • Elements: a Regression Set / a benchmark version / before-and-after comparison on every change
  • See: Chapters 9 and 12
  • If unmet:
    • No Regression Set → every change is a gamble, and a week of changes breaks the system
    • No benchmark version → nobody can tell improvement from regression
    • No release gate → anyone can change anything, and when it breaks there's no one to hold accountable
  • Checklist: [ ] Regression Set at least 200 cases [ ] baseline version frozen and recorded [ ] automated before/after comparison on every change [ ] change log traceable

• ★ Wall Six: trustworthy results → a chain of evidence

  • Contents: when the customer asks "where did this number come from" or "why should I believe this conclusion," you can produce the evidence
  • Elements: execution records per call / original inputs / intermediate reasoning steps / the basis for the final decision
  • See: Chapters 9 and 13
  • If unmet:
    • No execution records → a challenged result can only be rerun, never traced
    • Incomplete execution records → the customer sees the result but not the reasoning behind it
    • No archived raw data → once the data changes, a post-mortem becomes impossible
  • Checklist: [ ] every call's input logged [ ] intermediate steps (retrieval results, model output) inspectable [ ] the basis for the final decision showable to the customer [ ] log retention period defined

• ★ Wall Seven: cost and stability → cost attribution + a budget circuit breaker

  • Contents: how the bill gets computed, what happens when it's exceeded, how the system meets its Service Level Objectives (SLOs)
  • Elements: per-call cost attribution / cost alerts / automatic degradation when the budget is exceeded
  • See: Chapters 9 and 14
  • If unmet:
    • No cost attribution → the month-end bill explodes and the customer is furious
    • No alerts → by the time anyone notices, spend is running at several times the budget
    • No automatic degradation → expensive calls keep running while costs are already high, a vicious cycle
  • Checklist: [ ] cost model documented [ ] average cost per 1,000 calls measured [ ] budget-alert thresholds set [ ] automatic degradation on budget overrun implemented

• ★ Wall Eight: transferable responsibility → a responsibility handoff table + a runbook

  • Contents: who is responsible when something breaks, and who the system gets handed to once the FDE leaves
  • Elements: incident classification / first responsible party / escalation path / operations documentation
  • See: Chapters 8 and 17
  • If unmet:
    • No responsibility table → when something breaks there's nowhere to point a finger, and the relationship breaks
    • The runbook lives only on the vendor's side → the customer can't take over
    • No escalation path → an alert fires in the middle of the night and there's no one to reach
  • Checklist: [ ] responsibility matrix (incident type × responsible party) drawn up [ ] escalation timing and escalation target explicit [ ] the customer has reviewed the runbook [ ] at least one incident drill run

Non-critical items (check them; phasing is acceptable)

• Approval tickets for high-risk actions

  • Contents: a human approves before any write operation; run_id / approver_id / action details recorded
  • See: Chapter 11

• Service Level Objective (SLO) commitments on three dimensions (availability / latency / quality) + a degradation path

  • Contents: the metrics written into the Service Level Agreement (SLA) contract, defined across the three dimensions
  • See: Chapter 14

• Risk-tier table (read-only / reversible / irreversible / outbound)

  • Contents: classify every operation by risk tier
  • See: Chapter 11

---

Phase IV: Launch and Adoption (Chapters 8, 15, and 16)

Phase goal: results proven, system stable. The last gate from trial to production: the customer has to dare to take it, and employees have to want to use it.

Critical items (★)

• ★ The outcome acceptance sheet (metric + baseline + window + denominator definition + who decides, frozen in advance)

  • Contents: all five elements complete — print one copy and pin it in the review room
  • Elements:
    • Baseline value (the manual process's measured level, from Chapter 6)
    • Target value (the level to be reached)
    • Measurement window (weekly / monthly)
    • Denominator definition (who counts, who doesn't)
    • Who decides (one person's name, not "the business side")
  • See: Chapter 8
  • If unmet:
    • Metric not frozen in advance → after launch the metric gets changed because "this month's data is a special case," and the bickering never ends
    • Murky denominator → the customer says "the target wasn't met," the vendor says "you're counting wrong"
    • No named judge → nobody can persuade anybody, and the project hangs unresolved
  • Checklist: [ ] all five elements written into the contract [ ] the judge has signed [ ] rollback lines for each stage of the gradual rollout quantified

• ★ The operational readiness checklist (someone watches the alerts / degradation works / someone has actually read the runbook)

  • Contents: four checks — not one can be skipped
  • Elements:
    • Someone watches the alerts: when one fires, it maps to a name on the on-call roster
    • A working degradation path: including the fallback to Excel and manual work, and it has been rehearsed
    • A runbook: the customer holds a copy and has read it through
    • An on-call schedule: who covers the first week after launch and the committed response time, in black and white
  • See: Chapter 8
  • If unmet:
    • Alerts fire with no one watching → the customer finds out about the outage first
    • No degradation path → every system failure is a business outage
    • The runbook lives only on the vendor's side → a middle-of-the-night failure with no one to take the customer's call
    • A blank on-call sheet → who handles the first alert?
  • Checklist: [ ] the on-call roster has full coverage [ ] the degradation procedure rehearsed once [ ] the customer has confirmed receipt of the runbook and read it [ ] the on-call Service Level Agreement (SLA) on response time signed

• ★ The responsibility handoff table (a first responsible party for every incident scenario)

  • Contents: list every way the system could die, one by one, writing in the first responsible party and the escalation path for each
  • Example:

    What Went Wrong	First Responsible Party	Escalation Path
    Model output error causing business loss	Who judges it, who blocks it, who pays for it	To both sides' leads within 1 hour
    Data source failure / upstream outage	The data-side point of contact	To vendor on-call within 4 hours
    Incident caused by user error	Customer-side system administrator	To a joint post-mortem, same day
    Cost overrun	The platform provider	To the project lead, in the weekly report

  • See: Chapter 8
  • If unmet:
    • No responsibility table → when something breaks, "this isn't our problem" ←→ "how could this possibly be our problem," and the relationship breaks
    • No escalation path → a middle-of-the-night failure with no one answering, and the incident escalates into a customer complaint
    • Legal liability written down but no handling procedure → names on paper, no one picks up in reality
  • Checklist: [ ] at least 5 plausible failure scenarios listed [ ] a first responsible party named for each (a person's name, not a department) [ ] escalation timing and escalation target explicit [ ] signed by both sides

Non-critical items (recommended, phasing acceptable)

• Gradual rollout plan (staged traffic increase + a rollback line at each stage)

  • Contents: first 10% of users → 20% → 50% → 100%, with explicit rollback conditions at every stage
  • See: Chapter 8

• ★ The three incentive questions answered (quick reference: did the scorecard change / who keeps the time saved / whose problem is a mistake)

  • Contents: why users would use this system at all
  • See: Chapter 15

• Champion network (one champion per team)

  • Contents: in every affected department, one person who understands the system and can answer colleagues' questions
  • See: Chapter 15

• Activation metric definitions (activation rate / diversion rate, with the denominator and window pinned down)

  • Contents: how to count a user as genuinely using the system
  • See: Chapter 15

---

Phase V: Handoff and Flowing Back (Chapters 16, 17, 21, and 22)

Phase goal: pass the baton cleanly from the FDE to the customer; distill what this delivery produced into reusable product capability.

Critical items (★)

• ★ Handover of the three assets

  • Contents: three things the customer must catch (miss one, and the handoff doesn't yet count as successful)
  • Item one: the operations manual
    • Includes: a fault-troubleshooting tree, a Q&A for common questions, the on-call procedure, and the manual-fallback steps
    • Customer acceptance standard: at least one person can work through one real incident using the manual
    • See: Chapter 17
  • Item two: the eval set + maintenance responsibility
    • Includes: the eval set (for ongoing verification) + a plan for replenishing it on a schedule (and who owns that)
    • Customer acceptance standard: they can run the evaluation themselves and know how to add new samples
    • See: Chapter 17
  • Item three: the incident playbook
    • Includes: diagnosis and handling steps for the common failures, each with a worked example
    • Customer acceptance standard: they have seen the scenarios in the playbook and know how to handle them
    • See: Chapter 17
  • If unmet:
    • The manual never rehearsed → the first real incident sends them straight back to the FDE
    • The customer doesn't know how to maintain the eval set → the system drifts and nobody notices
    • No incident playbook → every failure becomes an emergency call for help
  • Checklist: [ ] all three documents received and reviewed by the customer [ ] each one rehearsed once (not "seen" — "done") [ ] the customer can independently handle the 3 most common failures

• ★ Every customization request tagged (one-off / industry-common / general-purpose)

  • Contents: of this project's requirements, which belong only to this customer, which are industry-common, and which can be distilled into product
  • See: Chapters 21 and 22
  • If unmet:
    • Everything treated as a "one-off requirement" → the next customer starts from zero again
    • Everything promised as "industry-common" → the promise is too heavy and later projects' costs explode
    • Nothing tagged → at the feedback review, nobody knows what to distill
  • Checklist: [ ] every customization requirement labeled with a category [ ] one-off requirements carry an explicit sunset date [ ] first-pass code for the general-purpose capability committed

Non-critical items (recommended)

• The handoff staircase executed (run alongside → shadow → exit, with a time frame and exit criterion at each step)

  • Contents: stage one, the FDE runs alongside; stage two, the FDE shadows; stage three, the customer runs it alone
  • See: Chapter 17

• Customer-side operators pass a hands-on check (not just handed documents)

  • Contents: the customer's point of contact has actually operated the system, hit problems, and solved some on their own
  • See: Chapter 17

• At least one feedback review (delivery + product, both seats filled)

  • Contents: after the project closes, the delivery side and the product side sit down together and sort out what capability from this engagement is worth keeping
  • See: Chapter 22

• Customization trend tracked (customer N vs customer 1)

  • Contents: whether the customization load is rising or falling — the product-market-fit indicator
  • See: Chapter 21

---

Usage Recommendations

For FDE teams

• Phases 1–3: score the project against the checklist at kickoff, marking each item ✅/⚠️/❌
• Phases 4–5: two weeks before launch, bring the customer in and walk the list item by item, closing out every ★-marked must-have
• Ongoing maintenance: after each project, run a retrospective and add the pitfalls this project surfaced

For customer-side procurement

• Write the "critical items" section of this checklist into your procurement requirements
• Cross-reference Chapter 8's customer self-check list against the critical items here, and ask whether your procurement process itself manufactures the "look but don't touch" trap

For project managers

• Early assessment: use the checklist to gauge quickly how far a "PoC complete" project still is from production
• Risk management: turn the checklist into a Gantt chart, and the effort and dependencies behind each of the eight walls become visible at a glance

---