vendor-delivery-and-service-qualityrun.md02_define/output/spec.mdScope: .icm/runs/vendor-metrics/01_scope/output/scope.md (M3, M6; BR-10, BR-13, BR-15, BR-17,
BR-18, BR-19, BR-20, BR-23; Q-2 and Q-7 answered by Paul).
Stub: .icm/intake/vendor-metrics/vendor-delivery-and-service-quality.md (3 of 10).
Foundation: metric-dictionary-and-claim-surface (stub 1, merged) — the claim surface, the period
resolver, the seeded M1–M10 definition records and the tenant-scoped query layer already exist.
The vendor dashboard is the evidence for a vendor's subscription, and two of the numbers on it cannot currently be defended. Delivered depends on the final bill settling as well as the work finishing, so a payment delay reads as a delivery slowdown and nobody can tell the two apart — and the platform records neither the customer's acknowledgement that the work is done nor a settled delivered date, so there is no fact to date the claim from. Separately, the dashboard promises a "service-quality breakdown" that cannot exist: a customer answers exactly one satisfaction question, so there are no sub-scores for communication, timeliness or expertise to break down, and showing one would invent detail the customer never gave. Meanwhile the existing dashboard reports "% within SLA" against a target that has nothing to do with the platform's configured SLA object.
Stub 1 landed the claim surface and reported no figures through it. This stub is the first metric family to go through it, and it is the one that decides what "Delivered" means for the four later stubs that hang off it (M4 satisfaction, M7 retention, M8 active service revenue, M10 revenue at risk). Getting it right is what Refine the bridge / Q2-2026 Objective 1 (Establish Product-Market Fit with Vendor Partners) rests on: a vendor who catches one wrong number stops believing all of them, including the ones that make the renewal case.
Two metric families — M3 Revenue Delivered and M6 On-Time Delivery Rate — rendered as claims through the existing surface, plus the delivery facts they have to be derived from, captured rather than approximated.
An engagement (a lead with an accepted proposal) is Delivered when all three hold (Q-2):
The delivered date is the day the last of those three conditions was met. The last-milestone-accepted date — the day condition 2 completed — is carried alongside it everywhere the figure is read, so a payment delay never reads as a delivery slowdown.
The lead workflow's delivered status is the operational signal that work has finished; it is not
the basis date and does not by itself make an engagement Delivered. That is what the scope's edge
case requires: an engagement delivered but never paid has not reached Delivered, and appears in
active service revenue as complete and awaiting payment instead.
Customer acknowledgement is captured in this stub (Jamie's ruling, interrogation 2026-08-17 — the measure is honest from day one, never approximated). It is recorded as an additive fact on the milestone — acknowledged-at, acknowledged-by, and whether it was recorded on the customer's behalf — not a new milestone status and not a gate on any existing flow. Nothing in delivery, invoicing or the workflow engine starts depending on it; what changes is that the fact now exists.
No backfill. Nothing historic is treated as acknowledged. Engagements that are otherwise complete but carry no acknowledgement appear in an explicit awaiting customer acknowledgement bucket with their count, never silently excluded and never counted as delivered (BR-17). Revenue Delivered will read thin until acknowledgements accumulate; that is the honest state of the records, and the surface says so rather than inventing a number.
The seeded M3 record already names it Revenue Delivered; today it would render a count under a money heading, which is the exact failure this scope exists to prevent.
The metric is renamed. "SLA Success Rate" is rejected: the platform has a real, configured SLA
object with its own types and targets (apps/web/app/(app)/admin/settings/sla/), and a vendor KPI of
that name measuring the agreed date instead would be one concept with two values on the very
dashboard this scope is cleaning up. M6 becomes On-Time Delivery Rate, and the screen states that
the measure is the agreed date. The three other definition records that reference the old name — M3's
and M7's related-KPI links, and M4's why-it-matters line — are corrected in the same change, so no
surface can still show it.
Headline: the share of engagements delivered in the period that were delivered on or before the agreed date.
The agreed date is the accepted proposal's end date, shifted by the signed timeline impact of every change-control entry agreed before delivery. An agreed change resets the date measured against, and the count of engagements whose date changed is shown beside the headline.
An engagement with no agreed date recorded goes into its own explicit bucket with its count, and out of both the numerator and the denominator (BR-17). It is never defaulted to on-time.
Four counts over the same delivered engagements replace the impossible "service-quality breakdown" (Q-7 — "YES V1"):
| Measure | Fact it counts |
|---|---|
| Delivered on time | Delivered on or before the agreed date |
| Delivered to plan | No change-control entry agreed against the engagement — the plan it was sold on is the plan it ran |
| Escalation-free | No flagged requirement, no queried bill, and no escalated blocker |
| Billed without query | No bill against it was ever rejected or disputed |
Each is a plain count with its share, each opens to its own records, and the four are reported beside satisfaction and never merged into a satisfaction score (BR-15).
The screen states the distinction: satisfaction is what the customer said; delivery quality is what happened. They are allowed to disagree.
No sub-score for communication, timeliness or expertise appears anywhere.
Each measure reads a record the platform already keeps, or one this stub starts keeping. Nothing is approximated.
| Fact | Source of truth |
|---|---|
| Milestone complete | milestone.status = "done" |
| Customer acknowledged | new — acknowledgement fields on milestone, written by the customer or recorded on their behalf |
| Bill settled | invoice.paid / paidAt |
| Bill queried | new — a durable queried fact on invoice, stamped when it is rejected or disputed |
| Agreed change (date) | change-control.timelineImpactDays |
| Agreed change (scope) | the existence of a change-control entry — an agreed, recorded change to the plan |
| Flagged requirement | lead.brdFlag / brdFlagReason |
| Manager intervention | blocker.escalated |
"Delivered to plan" uses change control as its source, not an approximation of one. The stub warned that scope changes need a source of truth before the measure is honest; a change-control entry is that record — an explicitly agreed change carrying its own timeline and cost impact. Any entry means the plan changed. Nothing infers a scope change from anything else.
"Billed without query" needs a durable fact because invoice status history is not kept: an
invoice disputed and later paid shows only invoice_paid today. The queried moment is stamped on the
invoice when rejectInvoice/disputeInvoice runs, and backfilled for invoices currently sitting in
a rejected or disputed state. Queries raised and resolved before this ships are not recoverable;
the surface states that the measure runs from the day the fact was first recorded rather than
implying a clean history.
M3 and M6 render as Claims through the existing claim surface on /vendor/metrics — headline,
at most two caveats, and a RecordSet with its reconciliation lines. Being the first figures on the
surface, they also land the obligations stub 1 deferred to the first metric family: the read's as-at
moment and its freshness statement against the 15-minute ceiling, sourced once per read (BR-20). The
period comes from the existing resolver; nothing re-derives one.
The tenant scoping stays in the query layer the surface owns, so a vendor reaches only their own engagements and customers by every route including the drill-downs (BR-23).
packages/services/src/db/models/{milestone,invoice}.ts — the acknowledgement and queried facts,
with their migrations under packages/services/src/db/migrations/ (index + backfill).packages/services/src/db/migrations/ — a definition-correction migration for M3 and M6 and the
three records that name the old M6.packages/services/src/db/services/milestone/ — the acknowledge write path.packages/services/src/db/services/invoice/ — stamp the queried fact at reject and dispute.packages/services/src/db/services/vendor/ — the delivered-engagement read, the two claims and
their record sets.apps/web/app/(app)/vendor/metrics/ + apps/web/components/vendor/metrics/ — figures on the grid,
the drill-downs, the satisfaction-versus-delivery statement.apps/web/components/dashboard/customer/ — the customer's acknowledge affordance on the milestone
progress surface, and the on-behalf-of route for CSM/SDM./vendor/delivery. Those keep their current values and definitions; stub 10
(vendor-dashboard-reconciliation) brings them onto these definitions. This stub deliberately
leaves the gap visible rather than doing stub 10's job — the same call stub 1 made.apps/demo — frozen 2026-08; its mock vendor-metrics.ts keeps the old M6 name and is not
updated.03_build/output/notes.mdfeat: vendor-delivery-and-service-quality — M3 and M6 from delivery factsThe facts first — both gaps the spec named were real, and neither is approximated.
packages/services/src/db/models/milestone.ts — acknowledgedAt, acknowledgedBy and
acknowledgedOnBehalf. A fact recorded beside the delivery flow: no status moves, no
transition is asserted, and nothing in delivery, invoicing or the workflow engine reads it.packages/services/src/db/models/invoice.ts — queriedAt, stamped once at the first
reject/dispute and never cleared. Invoice status carries no history (statushistories is
entityType: "lead" only), so without this a bill disputed on Monday and paid on Friday reads
clean..../services/invoice/index.ts — the stamp, inside the existing $set on both
rejectByManager and disputeByManager, conditional on the bill not already carrying one..../services/milestone/index.ts — acknowledge(). It resolves the owning engagement through
the milestone's proposal to its lead, which serves two purposes: the customer's own scope is
enforced there rather than in each caller, and the caller gets the lead id back to invalidate
the right cache.permissions/{registry,defaults}.ts — milestone.acknowledge: customer at own scope, csm and
sdm at full (recording for a customer they manage). The expert is deliberately excluded —
the party that did the work is not the party that accepts it.Three migrations.
…-milestone-acknowledge-role-templates — grants the new permission to already-seeded tenants,
at the same scopes defaults.ts holds. Without it the write path is unreachable in every
existing tenant.…-invoice-queried-fact-backfill — stamps the bills still sitting rejected or disputed, each
from its own updatedAt via an update pipeline. Recovers only what is recoverable: a bill
queried and resolved before this shipped left no trace, and nothing invents one.…-delivery-metric-definitions — rewrites M3's and M6's records, and the three that name M6.
$set, not $setOnInsert: the records exist and are wrong, so create-if-missing would leave
every deployed tenant reading the old text.The metrics.
packages/services/src/shared/vendor-delivery.ts — the pure logic, in /shared so the
definition of Delivered is unit-tested with no database near it. Classification, the agreed
date, the four measures, the trend, the record sets and the two claims.packages/services/src/shared/vendor-delivery.test.ts — written from the acceptance criteria:
each of the three conditions in turn, the payment-delay case, the no-agreed-date bucket, each
measure's source, whole-unit money, and the empty-vendor case..../metric-dictionary/vendor-delivery.ts — the read. Leads scoped through vendorClaimFilter,
then proposals/quotes/milestones/invoices/change-control/blockers by $in on those ids.apps/web/lib/vendor-delivery-figures.ts + /vendor/metrics/page.tsx — both claims on the
surface. The two families are read independently, so a failure in one leaves the other reporting.apps/web/components/vendor/metrics/claim-figure.tsx — two new breakdown modes (matchValue
with an explicit cohortKey, and truthy) plus a cohortNoun. byReason is untouched and now
shares one implementation with matchValue rather than being forked.apps/web/components/milestones/** — the acknowledgement affordance and its action, in a
neutral home because both the customer workspace and the staff workspace use them.whyItMatters, and no screen adds it to another.monthBucket, the measure columns), not a re-parsed label.awaitingSettlement and appears in neither figure. Unit-tested.Claim contract, which cannot be built without a record set.vendorClaimFilter, which throws without a vendor id.Decisions taken while building, that the spec did not settle:
paidAt existed is still
settled — it just contributes no date, and the engagement falls back to its acceptance day.
Filtering on the date would have silently dropped delivered engagements.Two bugs caught and fixed before pushing, both worth a second look:
queriedAt with $min. BSON orders null below every date, so $min
against a defaulted-null field would have kept the null and the fact would never have been
recorded. Replaced with a conditional $set.trendMonths first stepped with setUTCMonth, which lands on the wrong month whenever the
anchor day does not exist in the target month — a period ending on the 31st would have skipped a
month and repeated another. Replaced with plain month arithmetic, and unit-tested for twelve
distinct months.What the preview will and will not show. The four measures read facts that mostly did not exist until this change: no milestone carries an acknowledgement, so on a fresh preview every otherwise complete engagement sits in "awaiting customer acknowledgement" and Revenue Delivered reads zero until someone acknowledges one. That is the designed behaviour — the honest state of the records — and it is the single most important thing to exercise: acknowledge every milestone on a completed engagement as the customer, and the engagement should appear in both figures.
Index note for Verify. The delivery read filters {tenantId, vendor} on leads and then $in
on ids for the six related reads. invoices already has {tenantId, lead, status, createdAt} and
milestones {tenantId, proposal, order}, which serve theirs. As with stub 2, the lead read has
no {tenantId, vendor} index and seeks the tenantId prefix — deliberately not added without
evidence.
Unbounded by design, and worth a reviewer's eye: the lead read is not date-filtered, because the delivered date is derived and there is no stored field to filter on. It is bounded by one vendor's own leads within one tenant. If that becomes a volume problem the fix is a stored delivered date, which is a different change from this one.
Context budget: within the Inputs table, plus the stub-2 surfaces this family attaches to
(claim-figure.tsx, vendor-funnel.ts as the pattern to follow, the metric-dictionary service)
and the delivery records the spec's touches: names.
05_ship/output/changelog.mdTwo more figures are now live on your dashboard and in the KPI dictionary, and both hang off the same question: what does it actually take for a piece of work to count as delivered?
An engagement counts as delivered when three things are true: every milestone is complete, the customer has acknowledged each one, and every bill against it is settled. That middle condition is new. Until now the platform had no record of a customer agreeing that a piece of work was done — only of the team saying so.
Customers can now acknowledge a completed milestone from their own project view. It changes nothing about the project: no status moves, nothing unblocks, and no invoice waits on it. If a customer does not use the portal, a CSM or SDM can record the acknowledgement for them, and the record shows which of the two it was.
The contracted value of the engagements that reached delivered in the period, with the count of those same engagements beside it. Both come from one list of records, so the value and the count cannot disagree — open the figure and the engagements are there, adding up to exactly what the headline says.
Because delivered waits on the final bill, every row carries two dates: the day the last milestone was accepted, and the day the engagement was delivered. A payment that lands a month late moves the second date and not the first, so a slow payer never reads as a delivery slowdown. Work finished in one month and billed in the next counts in the later month, and the earlier month is not restated.
Twelve months of history sit behind it, month by month, on the same records.
The share of those engagements delivered on or before the date agreed with the customer. An agreed change to the delivery date resets the date measured against, and the number of engagements whose date moved is shown next to the figure. An engagement with no agreed date recorded sits in its own bucket with its count, rather than being quietly treated as on time.
Behind it are four counts over the same engagements — delivered on time, delivered to plan, escalation-free, and billed without query. Each is a plain count of something the platform records, and each opens to the engagements behind it.
These four are what happened. Satisfaction is what the customer said. They are reported side by side and never merged into one score, and they are allowed to disagree — when they do, that gap is usually the most useful thing on the page.
Nothing historic has been marked as acknowledged. Engagements that are otherwise finished but carry no acknowledgement are shown in their own group with a count, so the figure starts thin and fills in honestly as acknowledgements arrive, rather than opening with a number nobody agreed to.
And "billed without query" runs from today. A bill that was queried and then resolved before this release left no record behind it, so the measure says which date it counts from rather than implying a clean history.
05_ship/output/investor-update.mdWho it's for: Vendors, SDMs, Customers What shipped: Revenue delivered and on-time delivery rate are live; customers can acknowledge a completed milestone. Why it matters: Refine the Bridge — Q2-2026 Objective 1: Establish Product-Market Fit with Vendor Partners. Delivered now needs the customer's agreement, not ours.
Dig deeper: https://github.com/sustentus/sustentus/pull/857 · https://help.sustentus.com/changelog/2026-08-19-vendor-delivery-and-service-quality
05_ship/output/release.md9430431 (the code): Quality Project ✅, Migrate preview database ✅ (all
three of this run's migrations applied cleanly against the preview database), Audit database
✅, Project run labels, Spec structure, Intake structure, Ship completeness and Vercel Preview Comments ✅; Vercel web and marketing Ready. The docs/changelog/ship-note commits
re-ran the same set, and the merge was taken only on that round reporting green, checked once,
never on red. Migrate production database skipped (merge-only) and
Review diff against CONVENTIONS.md skipped (flag off). One earlier round failed and was fixed
rather than re-run: f24353c, six unit tests, two causes — a test factory that left
completedMilestones at its default when a test overrode milestoneCount, so five engagements
read as unfinished work; and M6's caveat containing the string "SLA" (as a denial), which its own
test bans outright. The copy was wrong, not the test.technical/packages/services — the Metric families bullet now names the
second family: shared/vendor-delivery.ts (the three conditions that make an engagement
Delivered, and both figures derived from one list of records) and vendorDeliveryService reading
through the same vendorClaimFilter guard.business/service-journey/delivery — the three acknowledgement fields in the
milestone table, plus a Customer acknowledgement section stating that completing and
acknowledging are different acts by different parties and that acknowledgement gates nothing.
business/roles — Revenue delivered and On-time delivery rate under what a vendor sees; the
milestone acknowledgement under what a customer sees and cares about.
business/feature-role-matrix/projects — a milestone.acknowledge row (admin/csm/sdm full,
customer own, expert deliberately excluded) and the note explaining why..github/workflows/ship-note.yaml fires on this
merge and sends the committed file verbatim. Both Dig deeper links were filled before the
merge, not left as placeholders; body is 60 words against the 60-word cap.Every box below is checked against the code and the unit tests CI ran. None was checked against a running preview — see the gap section, which is the whole story of this run.
Claim contract, which cannot be constructed without a record setvendorClaimFilter, which throws without a vendor id (BR-23)Verify was not run. 04_verify/output/verify.md does not exist. Ship's own contract requires
it and sends a run back without it; this run went to Ship on Jamie's explicit instruction to
disregard Verify, and merged on his ticked box. That is recorded here rather than papered over,
because the boxes above would otherwise read as though a quality gate had passed them. It did not
run at all.
What that means concretely — none of these were performed by anyone:
/vendor/metrics rendering M3 and M6, the
drill-downs adding up, a customer acknowledging a milestone and the engagement then appearing in
both figuresWhat is evidenced: the unit tests CI runs (476 passing, 22 of them written from this spec's
acceptance criteria), the preview-database migration applying all three migrations cleanly, the
Vercel preview building the whole PR, and Audit database staying green with the two new fields.
The single most important untested behaviour: no milestone carries an acknowledgement, so on first load every otherwise-complete engagement sits in awaiting customer acknowledgement and Revenue Delivered reads zero. That is the designed behaviour, not a fault — but it has only been reasoned about and unit-tested, never seen.
{tenantId, vendor} index on leads — the query seeks the tenantId prefix and filters,
same position stub 2 recorded. Audit database is green; deliberately not added without evidence.vendor-money-states-and-active-service-revenue
owns the conversion work./vendor/delivery still shows the old "% within SLA" against a different definition. Left
deliberately, per the spec's Out of scope; vendor-dashboard-reconciliation (stub 10) retires it.Context budget: within the Inputs table, minus verify.md, which does not exist.