Run: service-fee-schema-and-data-removal

• branch: claude/service-fee-schema-data-removal-ixwspp
• pr: #746

Spec: Remove the service fee from the schema and from stored data

• slug: service-fee-schema-and-data-removal
• personas: Vendor, SDM, CSM, Customer, Admin
• touches: packages/services/src/db/models, packages/services/src/db/services, packages/services/src/db/migrations
• complexity: standard
• demo: none

Problem

service-fee-write-path-removal (#744) stopped the platform computing, storing and displaying the 10% service fee, but the concept is still load-bearing. serviceFee remains a persisted field on the quote and invoice schemas, six read paths still reconstruct money as total ?? (amount + serviceFee), and every document written before #744 still carries a fee — both in serviceFee and, invisibly, inside its total. Because every read prefers total, historic vendor revenue, SDM market signals, CSM revenue-at-risk and customer contract values stay 10% inflated with no field on the document left to explain why. Finishing the removal is the last step of the fee deprecation under Refine the bridge / Q2-2026 Objective 3 — Validate Technical Infrastructure & Payout Flow: the money the platform records must be the work value the expert quoted, on every record, old and new.

Proposed change

Delete the concept, then reconcile the data — schema change and migration in the same PR.

• Drop serviceFee from the quote and invoice Mongoose schemas and from the IQuote / IInvoice interfaces.
• Drop the quote model's totalAmount virtual — it exists only to express amount + serviceFee and has zero callers monorepo-wide.
• Simplify every money read from total ?? (amount + serviceFee) to total ?? amount, across vendor revenue flow and open revenue, SDM market signals, the CSM blocker queue, and customer project finance. Drop serviceFee from the VendorEnvelopeInvoice type and its projection — it is carried through the envelope and never read.
• Drop the now-dead $unset: { serviceFee: "" } from quoteService.updateFinancialsForProposal. It was #744's belt-and-braces cleanup; once the field leaves the schema Mongoose strict mode strips it from the update anyway, and the migration owns that cleanup instead.
• Ship a migration that, against the raw quotes and invoices collections, $unsets serviceFee and resets total = amount on documents where the field is present, so total means the same thing on every record.

Historic-data decision (owner-confirmed, the gate breakdown.md reserved): the migration resets total = amount as well as unsetting the fee. This rewrites a money field on already-issued quotes and invoices, and historic vendor revenue, SDM market signals, CSM revenue-at-risk and customer contract values will drop by the former fee percentage for pre-#744 records. That drop is the correct figure — the fee was never a real commercial term — and Ship must state it plainly in the release notes.

The migration is not symmetric and cannot be: once serviceFee is unset and total is reset, the original split is unrecoverable from the document. down is written as the best available inverse — recompute serviceFee = amount × 10 / 100 and total = amount + serviceFee — with a file header saying it reconstructs an assumed 10% rather than restoring the recorded value.

Acceptance criteria

• serviceFee and the totalAmount virtual appear nowhere under packages/, apps/web/, or apps/demo/ (apps/docs/archive/** and the shipped apps/help/app/changelog/** entry are excluded — they are historical records).
• IQuote, IInvoice and both Mongoose schemas no longer declare serviceFee; the quote model no longer declares the totalAmount virtual.
• Every money read in vendor (revenue flow + open revenue), sdm-market-signals, csm-blocker-queue and customer-project resolves as total ?? amount, and no query projection selects serviceFee.
• VendorEnvelopeInvoice no longer carries a serviceFee field.
• A migration exists in packages/services/src/db/migrations/ whose up runs a single aggregation-pipeline updateMany per collection against raw quotes and invoices, scoped to documents where serviceFee exists, setting total = amount and unsetting serviceFee.
• After the migration, no quote or invoice document has a serviceFee field, and every quote and invoice satisfies total === amount wherever amount is set.
• The migration's down is written, and its header states explicitly that it reconstructs an assumed 10% fee rather than restoring the recorded value, and that a backup of the quotes and invoices collections should be taken before running it in production.
• Vendor revenue, SDM market signals, CSM revenue-at-risk and customer project finance render unchanged figures for post-#744 records, and figures lower by the former fee percentage for pre-#744 records.
• pnpm format:check, pnpm lint and the per-app builds pass.

Out of scope

• Removing the total field itself, provably redundant once this lands — a separate leanness chore.
• Backfilling amount where it is unset; those documents fall back to total, leave them.
• Any replacement fee, commission or take-rate model — fees are settled outside the platform.
• Any change to apps/docs/archive/pipeline-runs/** or the shipped apps/help/app/changelog/2026-08-03-service-fee-write-path-removal/ entry.
• Docs and changelog updates for this run — Ship's job, in this same PR.

Open questions

• None. The one owner decision (reset total vs leave history) is settled above; Build implements the reset. Build should confirm the raw collection names are quotes and invoices before writing the migration — the models register as quote / invoice and Mongoose pluralizes, but the migration operates on raw collections and a wrong name would silently no-op.

Build notes: service-fee-schema-and-data-removal

• commits: see the run branch (claude/service-fee-schema-data-removal-ixwspp)
• demo: none (Design was skipped for this run)

What changed

• packages/services/src/db/models/quote.ts: dropped serviceFee from IQuote and the schema, and deleted the totalAmount virtual (amount + serviceFee) — it had zero callers monorepo-wide.
• packages/services/src/db/models/invoice.ts: dropped serviceFee from IInvoice and the schema.
• packages/services/src/db/services/vendor/index.ts: simplified both money reads (revenue flow and open revenue) to total ?? amount, dropped serviceFee from their query projections and inline row types, and removed the field from VendorEnvelopeInvoice, its projection and its mapping — it was carried through the envelope and never read.
• packages/services/src/db/services/sdm-market-signals/index.ts: quoteTotal is now total ?? amount; serviceFee gone from QuoteDoc and the .select(...).
• packages/services/src/db/services/csm-blocker-queue/index.ts: revenue-at-risk read simplified to total ?? amount.
• packages/services/src/db/services/customer-project/index.ts: totalContract simplified to total ?? amount; corrected the stale doc comment that described it as amount + serviceFee.
• packages/services/src/db/services/quote/index.ts: removed the now-dead $unset: { serviceFee: "" } from updateFinancialsForProposal. It was #744's belt-and-braces cleanup; with the field gone from the schema Mongoose strict mode would strip it from the update anyway, and the migration owns that cleanup.
• packages/services/src/db/migrations/1784400000000-remove-service-fee.ts: new — one aggregation-pipeline updateMany per raw collection (quotes, invoices), scoped to documents where serviceFee exists, resetting total = amount and unsetting serviceFee.

Acceptance criteria status

• serviceFee / totalAmount gone from packages/, apps/web/, apps/demo/ — post-edit sweep over packages/** (excluding dist/) returns no matches; apps/** was already clean after #744 (the only remaining hits are apps/docs/archive/** and the shipped apps/help/app/changelog/** entry, both excluded as historical records).
• IQuote, IInvoice and both schemas no longer declare serviceFee; the totalAmount virtual is gone from the quote model.
• Every money read in vendor (revenue flow + open revenue), sdm-market-signals, csm-blocker-queue and customer-project resolves as total ?? amount; no projection selects serviceFee.
• VendorEnvelopeInvoice no longer carries serviceFee.
• Migration exists with a single aggregation-pipeline updateMany per collection against raw quotes / invoices, scoped to { serviceFee: { $exists: true } }.
• No document has serviceFee, and total === amount wherever amount is set — cannot be confirmed here: the agent sandbox has no route to the database. The migration is written to produce exactly this; CI applies it (.github/workflows/db-migrate.yaml runs it against the preview DB on this PR and against production on merge). Verify should confirm from the migration check run, not from a local run.
• down is written, and the file header states in full that it reconstructs an assumed 10% fee rather than restoring the recorded value, and that a backup of both collections should be taken before running it in production.
• Figures unchanged for post-#744 records and lower by the former fee percentage for pre-#744 records — an observable outcome for Verify on the preview, not something Build can tick.
• pnpm format:check, pnpm lint, per-app builds — the factory's to report, not mine (per the Build contract these are not run locally). Read them off the PR's check runs.

Notes for Verify

• The migration is the risk in this PR. It rewrites total on already-issued quotes and invoices. That was the owner decision recorded in the spec (and in the scope breakdown.md), not a build choice — but it means historic vendor revenue, SDM market signals, CSM revenue-at-risk and customer contract values will drop by the former fee percentage for pre-#744 records. Ship must say so plainly in the release notes.
• One deliberate deviation from the stub's wording, worth a look. The stub said to reset total = amount wherever the two diverge. A blunt $set: { total: "$amount" } would remove total on documents whose amount is unset (an aggregation $set from a missing field drops the field), destroying their only surviving value and contradicting the spec's own "Out of scope: backfilling amount where it is unset — those documents fall back to total; leave them." The up therefore guards with $cond on $ne: ["$amount", null], so those documents keep their total and only lose serviceFee. Confirm that reading of $set-from-missing if you want the belt-and-braces check.
• The down filter is { serviceFee: { $exists: false }, amount: { $ne: null } } so a rollback can't double-apply a fee to a document that already has one.
• Collection names are the Mongo names (quotes, invoices) — the models register as quote / invoice via getModel, which calls mongoose.model(name, schema) and pluralizes. Worth a sanity check against the preview DB, since a wrong name would silently no-op rather than fail.

Verify: service-fee-schema-and-data-removal

• production-readiness: run — ready, no ❌. One ⚠️ found and fixed on branch (the down over-application, below). One operational carry-over from #744 restated for Ship.
• code-review: medium effort (spec complexity: standard), performed inline — the CI Claude review (Review diff against CONVENTIONS.md) reported skipped, so ENABLE_CLAUDE_REVIEW is off, and the /code-review skill is user-invocation-only so the agent could not launch it. Two findings: one fixed, one recorded. No CI review comments to triage (get_review_comments → 0).
• security-review: not required — the diff touches no auth, payment-provider, PII or route-policy code. It changes financial record semantics (a data concern, covered by production-readiness), not a security surface. No new routes, actions, env vars or client-supplied identifiers.
• playwright: TODO — manual DoD smoke performed instead.

CI on the head commit (all green)

Quality Project (format + lint + typecheck) ✅ · Migrate preview database ✅ · Audit database ✅ · Spec structure (advisory) ✅ · Project run labels ✅ · Vercel Preview Comments ✅ · Migrate production database skipped (correct — merge-only) · Review diff against CONVENTIONS.md skipped (feature flag off).

Vercel previews: web and marketing READY; demo, docs, help-centre, storybook Ignored — correct, the diff touches none of their paths.

DoD smoke (on the preview — each line says who verified it)

• serviceFee / totalAmount appear nowhere under packages/, apps/web/, apps/demo/ — demonstrated: full-tree sweep post-change returns no matches outside apps/docs/archive/** and the shipped apps/help/app/changelog/** entry, both excluded by the spec (agent)
• IQuote, IInvoice, both schemas and the totalAmount virtual — demonstrated: read in the branch diff; field and virtual both deleted (agent)
• Every money read resolves as total ?? amount, no projection selects serviceFee — demonstrated: all six sites traced in the diff (vendor revenue flow + open revenue, sdm-market-signals, csm-blocker-queue, customer-project) plus the two .select(...) projections and the envelope mapping (agent)
• VendorEnvelopeInvoice no longer carries serviceFee — demonstrated: type, projection and row mapping all dropped in the diff (agent)
• Migration shape — demonstrated: one aggregation-pipeline updateMany per raw collection, filtered on { serviceFee: { $exists: true } } (agent)
• Migration applies cleanly — demonstrated on the preview database, not self-certified (agent, from the Migrate preview database CI job log, quoted under "Migration evidence" below)
• down written, irreversibility + backup stated in the header — demonstrated: header now also states the two specific ways it is lossy (agent)
• No quote/invoice document has serviceFee, and total === amount wherever amount is set — the migration ran green against the preview DB, which is strong evidence it did what it says, but neither the agent nor CI queries the collections afterwards to confirm the resulting document state. Operator check, still outstanding.
• Vendor revenue / SDM market signals / CSM revenue-at-risk / customer project finance render unchanged for post-#744 records and lower by the former fee for pre-#744 records — operator, still outstanding: every one of these is behind Clerk sign-in and the agent has no preview credentials.
• auth: vendor · SDM · CSM · customer sign-in + dashboard reachable — operator, outstanding
• payments: not touched — no payment-provider, Stripe or payout code in the diff (agent)
• notifications: none expected — no notification type, template, preference rule or Ably publish is touched by the diff; sweep of packages/services/src/notifications shows no serviceFee reference (agent)

Migration evidence

From the Migrate preview database job on the head commit — the migration really ran against a database, it was not merely present in the tree:

Adding migration .../src/db/migrations/1784400000000-remove-service-fee into database
  from file system. State is down
up: 1784400000000-remove-service-fee
All migrations finished successfully

The workflow also guards the target: it refuses to run if MONGODB_URI / MONGODB_DATABASE_NAME resolve to sustentus-prod, so this was a preview database.

Findings & cleanup

• ⚠️ down invents a fee on records that never had one — fixed on branch. The rollback filter (serviceFee absent + amount set) cannot distinguish a document whose fee up removed from one written after #744 that legitimately never had a fee. Running down would apply a 10% fee to the latter and inflate their totals. The behaviour is inherent to a lossy inverse and the spec accepted a best-effort down, so the fix is honesty rather than logic: the file header now states this failure mode explicitly and recommends restoring a backup over running down. Also recorded there that the sweep is deliberately cross-tenant (a platform-wide concept retirement), so a future reader doesn't flag the missing tenantId filter as an oversight.
• Accepted — totalAmount removal is technically an API-shape change. schemaPlugin sets toJSON/toObject { virtuals: true }, so the virtual was serialized into every quote object (quoteService returns doc.toObject()). Removing it drops totalAmount from that payload. Accepted: a monorepo-wide sweep finds no consumer — the only totalAmount hits are unrelated mock strings in an apps/storybook Table story — and the apps/api named in the old comments does not exist in this repo. Flagging it because it is invisible in the diff.
• Checked, no issue — deploy/migration ordering. Removing the $unset: { serviceFee: "" } from updateFinancialsForProposal is safe in either order: a quote re-priced between the code deploy and the migration keeps a stale serviceFee that nothing reads, still gets total = amount from the $set, and is cleaned up when the migration passes over it. Nothing writes the field any more, so no new ones appear.
• Checked, no issue — up preserves total where amount is unset. $ne: ["$amount", null] is false for a missing field, so those documents fall through to "$total" and keep it, matching the spec's "leave them" instruction. up is also idempotent: the $exists filter makes a re-run a no-op.
• Carry-over for Ship, from #744 — not this PR's change. SERVICE_FEE_PERCENTAGE still needs unsetting in the Vercel dashboard. Nothing reads it (it is absent from turbo.json, docs/ENV.md and all code), so it is inert rather than dangerous, but it should not outlive this scope.
• Note for Ship — the honest release-note line. Historic vendor revenue, SDM market signals, CSM revenue-at-risk and customer contract values will drop by the former fee percentage for pre-#744 records once this merges and the production migration runs. Expected and correct, but it must be stated plainly rather than left to be discovered.

Run copy of the published entry: apps/help/app/changelog/2026-08-03-service-fee-schema-and-data-removal/page.mdx

Title: Older quotes and invoices now show the work value too Personas: customer, expert, vendor, csm, sdm, admin

Covers: the migration that cleared serviceFee from stored quotes and invoices and reset each total to its amount, and the resulting downward shift in historic revenue figures. Written as the follow-on to #744's entry, which told customers older records would keep their issued totals — this entry is what changes that.

Every quote and invoice now records the expert's price, past ones included

Who it's for: Vendors, SDMs, CSMs, customers and admins What shipped: The retired 10% service fee has been cleared from stored quotes and invoices, and each total reset to the amount the expert quoted. Why it matters: Refine the Bridge — Q2-2026 Objective 3: Validate Technical Infrastructure & Payout Flow. Money means one thing on every record.

Historic revenue, market signals and contract values drop by the former fee's share: the fee was folded into those totals and no longer inflates them.

Dig deeper: https://github.com/sustentus/sustentus/pull/746 · https://help.sustentus.com/changelog/2026-08-03-service-fee-schema-and-data-removal

Ship: service-fee-schema-and-data-removal

• pr: #746 · merged: yes — 2026-08-03, squash as 0940e76a
• CI: green — Quality Project, Migrate preview database, Audit database, Spec structure, Project run labels, Vercel Preview Comments all ✅; Migrate production database skipped (merge-only, runs on this merge); Review diff against CONVENTIONS.md skipped (flag off)
• technical docs: no technical docs impact — nothing in apps/docs/app/technical/** describes the serviceFee field or the quote/invoice money model
• business docs: no business docs impact — #744 already removed every service-fee reference from business/**; a sweep of apps/docs/app outside archive/ returns no matches
• release notes: both
• sent: ship note sent to #product-update on 2026-08-03

Acceptance check (vs spec)

• serviceFee / totalAmount nowhere under packages/, apps/web/, apps/demo/ — full-tree sweep clean outside apps/docs/archive/** and the shipped changelog entries (Verify, agent)
• IQuote, IInvoice, both schemas and the totalAmount virtual dropped — read in the diff
• Every money read resolves total ?? amount; no projection selects serviceFee — all six sites traced in the diff (vendor revenue flow + open revenue, sdm-market-signals, csm-blocker-queue, customer-project)
• VendorEnvelopeInvoice no longer carries serviceFee — type, projection and mapping dropped
• Migration is one aggregation-pipeline updateMany per raw collection, scoped to { serviceFee: { $exists: true } }
• Migration applies cleanly — ran green against the preview database on three separate commits (up: 1784400000000-remove-service-fee → All migrations finished successfully), proving it is also idempotent on re-run
• down written; header states the assumed-10% reconstruction, that it cannot distinguish a record whose fee up removed from one that never had a fee, and the backup requirement
• pnpm format:check, pnpm lint, per-app builds — Quality Project green on the head commit
• No document has serviceFee and total === amount wherever amount is set — the production migration runs on this merge. The preview runs are strong evidence, but the post-migration production document state was never queried. Still an operator check.
• Figures unchanged for post-#744 records, lower by the fee for pre-#744 records — never demonstrated on a signed-in preview; the agent has no Clerk credentials. Operator check.

Carried forward

• Unset SERVICE_FEE_PERCENTAGE in the Vercel dashboard. Inherited from #744, still outstanding. Nothing reads it (absent from turbo.json, docs/ENV.md and all code), so it is inert rather than dangerous — but it should not outlive this scope.
• Watch the production migration on this merge. Migrate production database was skipped on the PR by design and fires on the merge to main. It rewrites total on issued quotes and invoices, so it is worth confirming it completed rather than assuming.
• Two DoD lines above are unticked and stay that way until an operator confirms them. Shipping did not make them true.

Note

Shipped on the user's instruction with those operator checks outstanding — recorded here rather than quietly ticked, so the gap is visible in the durable record.