Run: storyline-full-coverage-generator

• branch: claude/storyline-full-coverage-generator-sjfpf7
• pr: #815

Spec: Extend the storyline contract and generator to light every surface

• slug: storyline-full-coverage-generator
• personas: Admin, CSM, SDM, Expert, Vendor, Customer
• touches: packages/services/src/db/services/demo-data, packages/services/src/db/services/demo-data/storyline, packages/services/src/db/services/demo-reset
• complexity: complex

Problem

The Phase 3 coverage matrix (.icm/intake/demo-data-quality/_source/audit-report.md) proves that whole surfaces open dark on the demo tenant after every reset: the invoice approvals queue, disputed and rejected invoices, quote draft/rejected and the awaiting_confirmation → quotation_process loop-back, a blocked milestone, expert evidence in any state, every persona's notifications, SDM outreach dispositions, the CSM day-over-day delta, and overdue/SLA-breached work. A demo that opens dark on the approvals queue or an empty notification bell cannot carry a vendor conversation, which is what Q2-2026 Objective 1 (Establish Product-Market Fit with Vendor Partners) needs the bridge to do. Separately, the generator hardcodes the tiered catalogue (platforms → products → services → skills) as module constants, so the ICP reframe that saas-vendor-storyline (stub 10) must perform cannot be done as a content change — which is the whole promise of the storyline contract.

Proposed change

Extend the Storyline contract and the generator so that mechanics stay in the generator and all content lives in the storyline module. Two halves, both landing in this run:

1. Content migration (no behaviour change).

• The tiered catalogue moves out of demo-data/index.ts module constants (CATALOGUE_PLATFORMS, UNIVERSAL_SERVICES, SPECIALIST_SERVICES, CATALOGUE_SKILLS) and onto the Storyline type as an authored catalogue block. The generator reads it; the default (Calderon) storyline carries today's values verbatim, so the seeded catalogue is byte-identical before and after this change.
• Per-engagement authored copy becomes expressible on the contract: StorylineEngagement and StorylineClosedEngagement gain optional per-record copy fields (proposal title/summary, milestone names, blocker label/impact, action-item labels, change description, customer/team messages, CSAT appreciation/suggestion). Where a record authors nothing, the generator falls back to the existing StorylineCopy pools exactly as today — so the Calderon storyline keeps working unchanged through the transition, and stub 10 authors the real values against a contract that already accepts them (D8).

2. New authored coverage, all deterministic under the existing deterministic.ts guarantees (hash-of-identity, never Math.random()), all tenantId-scoped, all idempotent:

Dark surface	What gets seeded
/finances/approvals queue	one engagement carrying an invoice_draft invoice
invoice chips	one invoice_disputed and one invoice_rejected invoice, on distinct engagements
quote states + loop-back	one quote_draft; one quote_rejected whose lead's status history runs … → quotation_process → awaiting_confirmation → quotation_process (the rejection loop)
milestone blocked	one blocked milestone on a work-in-progress engagement
expert evidence	rows in all four EXPERT_EVIDENCE_STATUSES (draft, submitted, approved, rejected); the rejected row carries a rejectionReason, reviewed rows a reviewedAt/reviewedBy
notifications	per-persona rows with a read/unread mix, authored type/title/body/link (types drawn from the existing typed enums)
SDM outreach	sdm_outreach_states rows covering all four OUTREACH_ACTION_STATES
CSM changeVsYesterday	one prior-day csm-portfolio-snapshot per CSM that differs from today's rollup by a storyline-authored delta
overdue / SLA breach	overdue action items (dueDate in the past, still open) and a backdated in-flight engagement whose status-history age breaches its SLA band

DemoDataResult gains a counter per new entity family so the admin demo-data page and the seed summary report what was written.

Decisions this spec settles

These follow from the code as it stands and correct or narrow the stub where it was written from the audit report rather than from the current source. Each changes what gets built.

• R1 — the snapshot is kept on reset, not purged. The stub asserts every newly seeded collection is already in the purge list. demo-reset/policy.ts classifies csm-portfolio-snapshot under KEPT_MODEL_FILES (deliberately — purging it blanked changeVsYesterday). expert-evidence, notification and sdm-outreach-state are correctly in PURGED_MODEL_FILES. Consequence: the snapshot write must be idempotent on its own, because reset will not clear it. The generator therefore seeds it through the existing csmPortfolioService.captureDailySnapshots(tenantId, snapshotDate), which already upserts on { tenantId, csm, snapshotDate }, rather than inserting rows directly. No change to policy.ts classifications is needed and none is made.
• R2 — a captured prior-day snapshot alone yields a zero delta. changeVsYesterday is group.atRiskNow - prior. Capturing yesterday from the world just seeded makes prior equal to today, so the delta renders 0 and the surface still tells no story. The storyline therefore authors the prior-day offset (an at-risk delta, with the matching RAG shift), which the generator applies to the captured prior-day row so the CSM portfolio shows real day-over-day movement.
• R3 — sla_definitions stay out. The gaps table lists the SLA config card as dark, but sla-definition is in KEPT_MODEL_FILES and is owned by the baseline db:seed tenant defaults, which survive reset. The demo seeder writing its own would duplicate a second writer into a collection reset deliberately preserves. Out of scope below.
• R4 — the lead pagination edge is unreachable under D7. ">20 leads in one kanban column" cannot hold at the same time as D7's "keep ~25 live leads" across nine stages. Notifications >50 and activity >25 are both reachable and are in. The lead-column edge is out of scope below.
• R5 — per-engagement copy ships as contract shape, values stay pool-derived. D8 permits this explicitly; stub 10 authors the values against the new world. This is what keeps the run one-PR-sized.
• R6 — one PR, not two. The stub asked Define to flag back if this is two-PR-sized. It is not, given R5: the content migration is a mechanical relocation whose output is byte-identical, and the new coverage is additive writers on top of it. Splitting would land a contract half whose new fields nothing reads. Building it as ordered separable commits (catalogue move → copy fields → one commit per entity family → tests) keeps it reviewable.
• R7 — notification volume follows D7's "where personas look daily". CSM, SDM and Expert get

  50 rows each (crossing the page boundary) with an unread mix; Admin, Vendor and Customer get a smaller populated set. Activity passes 25 rows for the personas whose feed is a daily surface.

Acceptance criteria

• The tiered catalogue is authored on the Storyline type; demo-data/index.ts holds no catalogue content constants, and seeding the default storyline produces the same platforms, products, services, skills and product↔service links as before the change (counts and names asserted in a test).
• StorylineEngagement and StorylineClosedEngagement accept per-record copy; a record that authors none falls back to the StorylineCopy pools, and the default storyline — which authors none — seeds an unchanged world.
• Every row of the coverage matrix's "confirmed seeder gaps" table has a seeded instance after a reset, except the two ruled out under R3 and R4: an invoice_draft, invoice_disputed and invoice_rejected invoice; a quote_draft and a quote_rejected; a blocked milestone; expert evidence in all four states; notifications for all six personas; sdm_outreach_states in all four action states; a prior-day CSM snapshot; overdue action items.
• The /finances/approvals queue, the /expert-evidence review queue, the SDM outreach controls and the notifications bell each render populated for their personas on the demo tenant after a reset (verified on the preview at Verify).
• The rejected expert-evidence row carries a rejectionReason; the quote_rejected lead's status history contains awaiting_confirmation followed by a return to quotation_process.
• CSM, SDM and Expert each hold more than 50 notifications with a mix of read and unread, so the notifications page paginates.
• The CSM portfolio's changeVsYesterday is non-zero for at least one CSM after a seed+reset.
• Two consecutive resets produce identical worlds — the existing determinism tests are extended to cover every new entity family — and re-running the seed without a reset creates no duplicates of them.
• Every new write carries tenantId; the existing cross-tenant reset test still proves a reset cannot reach another tenant's rows.
• An overdue engagement computes a red SLA band on both the expert work queue and the CSM blocker queue (unblocked: sla-stage-map-engine-alignment merged as #811).
• demo-reset/policy.test.ts stays green with no classification changes, and the reset result reports non-zero purged counts for expert-evidence, notification and sdm-outreach-state.
• DemoDataResult reports a count for each new entity family.

Out of scope

• The SaaS-vendor world content itself — saas-vendor-storyline (stub 10) authors it against this contract. This run authors only enough new content for the default (Calderon) storyline to light every surface.
• Authoring per-engagement copy values; only the contract shape and pool fallback ship here (D8, R5).
• Seeding sla_definitions — owned by the baseline db:seed tenant defaults and kept across reset (R3).
• The ">20 leads in one kanban column" pagination edge — irreconcilable with D7's ~25 live leads (R4).
• Zero-state demos — served by admin View-As bare-role mode; no filler personas (D11).
• Ably push, AI streams, Resend email and Blob uploads — interactive by nature; canned agent mode already covers demos.
• assertTransition enforcement for milestone and expert-evidence transitions — a real doctrine gap, explicitly not selected for this round (D1/D5); the seeder writes states directly as it does today.
• Any change to demo-reset/policy.ts classifications.

Open questions

• none

Context budget: within band. Read the stub, _source/decisions.md, _source/audit-report.md Phase 3, breakdown.md, and — as targeted confirmation of where things live — the storyline contract, deterministic.ts, demo-reset/policy.ts, the four target model schemas, and csm-portfolio's snapshot capture. No scope.md exists for this epic by design (the cut was made from _source/, per breakdown.md).

Build notes: storyline-full-coverage-generator

• commits: 075280d (contract + coverage + tests)

What changed

• demo-data/storyline/types.ts: added StorylineCatalogue (platform → product → service → skill), StorylineEngagementCopy, the invoice/quote state unions, StorylineEvidence, StorylineNotifications, StorylineOutreachState and StorylinePriorDay. Storyline gains catalogue, evidence, notifications, outreach and priorDay. Engagements gain copy, extraInvoiceStates, extraQuoteStates, blockedMilestone, overdueActionItems and staleDays. The evidence/outreach/notification enums are imported from the models rather than restated, so they cannot drift.
• demo-data/index.ts: the four catalogue constants are gone — the generator reads storyline.catalogue. Added a textFor(authored) resolver that the eight prose-writing helpers now take, so authored copy wins field by field over the pools. New writers for expert evidence, notifications, SDM outreach and the prior-day snapshot; new createOverdueActionItems; createLead accepts extraStatuses (the loop-back) and the delivery graph accepts blockedMilestone / extraInvoiceStates. Resolves the SDM and admin persona seats, which no storyline had ever attached anything to.
• demo-data/notification-fill.ts (new): the one piece of the new coverage with a decision in it — how a seat's list is padded, and which rows read as read. Pure, so it is unit-testable; the generator has no DB-free tier.
• csm-portfolio/index.ts: added captureBackdatedSnapshots(tenantId, daysBack). The tenant-local date derivation stays here, beside the one captureDailySnapshots already does, rather than being copied into the seeder.
• storyline/default.ts: the catalogue moved in verbatim; authored the evidence, notification, outreach and prior-day content, and tagged six engagements with the coverage flags.
• apps/web/.../demo-data-button.tsx: the summary table is a hand-kept list, so the five new counters were added or they would not render.

Acceptance criteria status

• Catalogue authored on the contract; generator holds no catalogue content — default.test.ts asserts the tiers are populated, every skill points at a defined service, every service is covered by a skill, and names are unique. The Calderon values moved across verbatim, so the seeded catalogue is unchanged.
• Per-record copy with pool fallback — textFor falls back field by field; an empty authored list counts as unauthored. Calderon authors none.
• Every gaps-table row seeded except the two ruled out under R3/R4 — asserted per family in default.test.ts.
• Surfaces render populated after a reset — needs the preview; this is a Verify-stage check by design (the criterion says so). Nothing asserts it here.
• Rejected evidence carries a rejectionReason; the rejected quote's lead returns to quotation_process via extraStatuses.
• CSM, SDM and Expert hold >50 notifications with a read/unread mix — padTo 56/54/52, asserted in both test files.
• changeVsYesterday non-zero — the storyline authors atRiskDelta: 2, applied on top of the captured prior-day row. Asserted non-zero in the storyline test; the end-to-end number is a preview check.
• [~] Two consecutive resets produce identical worlds — partially asserted. The pure fill logic has an explicit same-input-same-output test, and every new field derives from deterministic.ts (no Math.random(), no counters). But the generator itself cannot be unit-tested: the repo configures only the unit tier (node, no DB), so "reset twice and diff the collections" has no tier to run in. I did not improvise one — see Notes for Verify.
• Every new write carries tenantId — all five writers set it explicitly; the existing cross-tenant reset test is untouched.
• Overdue engagement computes a red SLA band — staleDays: 70 on Aldercroft. Delivery breaches at 35 business days (~49 calendar), so the test asserts staleDays > 49 rather than merely present; a smaller number would compute amber and the criterion would pass while the surface stayed orange.
• policy.test.ts green with no classification changes — none made. All three new collections were already in PURGED_MODEL_FILES with getters wired in demo-reset/index.ts.
• DemoDataResult reports a count per new family — five counters added, and surfaced on the admin page.

Notes for Verify

• The determinism criterion is the one to look at hardest. What is proven is that the logic is pure and key-derived; what is not proven by any test is that two real resets produce byte-identical collections. Two known sources of per-run variation already existed and still do: batch (a fresh ObjectId, in every quoteId) and allocateNextLeadRequestId. So "identical worlds" has always meant "identical content", not identical ids. Worth confirming on the preview by reseeding twice and comparing counts.
• The prior-day snapshot is the one write a reset does not clear. It is upserted, so a reseed cannot duplicate it — but if the offset block is ever changed to insert, a second seed would stack rows and the delta would drift. The updateMany moves atRiskNow and rag.red together and clamps both at 0.
• extraInvoiceStates wraps onto the milestone list (i % milestones.length) because a WIP engagement has two default invoices and four milestones. That is deliberate — an invoice with no milestone renders an empty approvals row — but it does mean one milestone can carry two invoices.
• Notification rows are written straight through the model, not the notifyX wrappers, so no Ably publish and no email fires for seeded data. That is what makes seeding them safe, and it is why the bell reads them fine.
• I did not run format/lint/typecheck/test locally (the hook blocks them) — read the Quality check run on the PR.

Context budget: within band. Read the spec, CONVENTIONS.md, packages/services/AGENTS.md, the demo-data and demo-reset sources named in touches:, and — to get the numbers right rather than guess — the four target model schemas, the notification type enums, sla/stage-map.ts + seed/tenant-defaults.ts (for the breach threshold behind staleDays), and csm-portfolio's snapshot capture.

Verify: storyline-full-coverage-generator

• production-readiness: run — 2 blockers, both fixed on branch; 4 advisories, 2 fixed, 2 routed to Ship (docs + changelog)
• code-review: high (spec complexity complex) — 7 findings, all verified against source, all 7 fixed on branch
• security-review: run — no HIGH or MEDIUM findings above the confidence bar
• playwright: TODO — manual DoD smoke required, and not yet performed

Status: NOT passed — the DoD smoke has not been done

Everything an agent can carry is done: three passes run, ten findings fixed, CI green (Quality Project success at c052466, covering format, lint, typecheck and the Run tests step). What is not done is the half of the Definition of Done that needs preview credentials and a signed-in browser. I have no Clerk session and no inbox, so I am not able to claim those lines, and the gate must not pass on unverified ones.

Three of the ten findings were defects that would have aborted the seed in production. Two more were surfaces that seeded successfully but could never be seen. None of them were visible to CI, because the repo configures only the unit tier — nothing exercises a real database write. That is the single most important fact about this run: a green suite told us almost nothing about whether this feature works.

DoD smoke (on the preview — each line says who verified it)

Agent-run:

• Contract/catalogue parity, per-record copy fallback, and all authored coverage traced in the diff and asserted in default.test.ts / notification-fill.test.ts (agent)
• Every new write carries tenantId; requireDemoTenant still precedes the first write (agent — read at demo-data/index.ts)
• All 26 notification links resolve to a real page under apps/web/app/(app) (agent — checked against the route tree)
• demo-reset/policy.ts unchanged; the three new collections were already purged with getters wired in demo-reset/index.ts (agent)

Operator-demonstrated — all outstanding:

• Press Populate on the demo tenant end to end. Report the wall time; the run got longer (evidence, ~180 notifications, outreach, snapshot) and the server action declares no maxDuration.
• Press Populate a second time. It must not throw. This is where the three duplicate-key blockers lived; the fixes are unverified against a real index.
• Run a full Reset, then confirm the reset result reports non-zero purged counts for expert-evidence, notification, sdm-outreach-state.
• /finances/approvals shows the invoice_draft row; disputed and rejected chips render.
• /expert-evidence review queue shows four rows, one per state, the rejected one with its reason.
• SDM outreach control shows populated cells across all four action states — the finding most worth re-checking, since it only works if the derived cells now carry a region.
• Notifications bell paginates for CSM, SDM and Expert (>50, mixed read).
• Customer workspace shows the blocked milestone on Meridian.
• Expert work queue (signed in as the expert persona) shows a red SLA band on Harbourline; CSM blocker queue likewise.
• CSM portfolio changeVsYesterday is non-zero.
• auth: each affected persona still signs in and reaches their dashboard
• payments: not touched
• notifications: none expected — seeded rows are written straight through the model, so no Ably publish and no email fires. Worth confirming no notification email arrived.

Findings & cleanup

Fixed on branch (0eb8332, c3bd196, 5c74d08, c052466):

1. Extra quotes reused the accepted proposal — quote.ts holds a global unique {lead, proposal}. The first quoting engagement aborted the entire Populate. Each extra quote now gets its own proposal, which also reads better: the rejected quote was priced against the proposal that had to be replaced.
2. Extra invoice states wrapped onto an already-invoiced milestone — invoice.ts holds a global unique {milestone}. Latent today (all extras sat on in_progress) but immediate for any storyline authoring one on a done stage. Stops instead of wrapping, and the storyline test now asserts extras only sit where a spare milestone exists.
3. Outreach rows inserted unconditionally against a unique cell index, so a second Populate threw after writing the leads and bells. Now upserted.
4. Outreach cells could never join. The surface derives cells from (platform, product, region, language) off each lead, and seeded leads never set location.region — so every cell was Unspecified and the authored coordinates matched nothing. Accounts now carry a region that reaches the lead, and the generator derives the coordinates from the world it just wrote; the storyline authors only the dispositions. This is a contract change made at Verify, and it is the right home for it: authoring a coordinate meant guessing which catalogue path the generator would pick.
5. staleDays sat on a non-persona expert. The expert work queue filters to the signed-in expert, so the red SLA band existed in the database where nobody demoing could see it. Moved to the expert persona's engagement.
6. 14 of 26 notification links pointed at routes that do not exist (/csm/blockers, /sdm/bids, /expert/projects, …) and would bounce off deny-by-default. All remapped to real pages.
7. Notifications and evidence were not idempotent — a second Populate doubled every bell. Both are create-if-missing now, matching the outreach fix.
8. ~180 sequential notification writes batched into one insertMany (safe: the plugin trio hooks queries and find, not save).
9. Delivery dates shift with staleDays, so a breached engagement no longer reads "72 days in delivery" beside a project that started three weeks ago.
10. readAt can no longer land in the future.

Accepted, recorded rather than fixed:

• blockedMilestone does not drive the health at-risk driver. customer-project/health.ts keys milestone criticality on blocksGoLive && overdue && status !== done; it has no blocked branch, and createMilestones never sets blocksGoLive. The milestone state seeds and the workspace icon renders, which is what this spec's acceptance criterion asks for — the stub's parenthetical about the health driver was never in the spec. Making health read blocked is a product change and belongs in its own scope.
• Overdue action items have no dedicated queue. The only reader is per-lead (listOpenByLead), so they surface on the lead rather than in a cross-account overdue view. The criterion ("overdue action items seeded") is met; a queue is new scope.

Routed to Ship, not done here:

• apps/docs/.../demo-environment/page.mdx is stale in three places: the "writes, in order" list omits all five new entity families; the catalogue is still described as generator-supplied; the append-only claim needs the idempotency nuance. Ship's docs-sync owns this.
• No changelog entry yet — in-repo precedent treats demo-world changes as user-visible. Ship's changelog-entry owns it.

Rollback note for the PR: no migrations, so code reverts cleanly. The prior-day snapshot step mutates csm_portfolio_snapshots, which a reset deliberately preserves — a reverted deploy does not undo the authored atRiskDelta shift. It is self-correcting (capture $sets the position wholesale before the delta is re-applied, and the row leaves the "yesterday" window next day) and demo-tenant-only.

Context budget: over band, deliberately. The reviews surfaced claims about live unique indexes and about which surface reads which field, and each one had to be checked against the model and the reading service before I acted — taking a review finding on trust is how a wrong fix lands. Files read beyond the Inputs table: quote.ts, invoice.ts, sdm-outreach-state.ts, sdm-risk/index.ts, customer-project/health.ts, lead.ts, the apps/web route tree, and the plugin trio.