Run: expert-quality-metrics

• issue: #469 # canonical home of the spec + state (labels, comments)
• branch: feat/469-expert-quality-metrics
• pr: #471

Stub: Expert quality metrics

• feature-slug: expert-quality-metrics
• epic: expert-dashboard-data
• personas: Expert (view); Customer (CSAT source); Vendor (vendor-rating source)
• initiative: Build the Bridge / objective: Q2 2026 Objective 3 — Validate Technical Infrastructure & Payout Flow
• depends-on: expert-workbench-foundation
• sequence: 8 of 8
• ⚠ flagged: carries two net-new concepts (vendor rating, rework). Split if either grows.

Problem

The quality half of the Performance section is dummy: csat, vendorRating, repeatWorkRate, reworkRate. Coverage is mixed — CSAT already has a model (csat.ts) and an avg-score helper (getExpertMatchingContext), repeat-work is derivable from the expert's leads, but vendor rating and rework rate have no backing concept at all.

Proposed change

• CSAT — reuse the existing per-expert avg-CSAT computation; map the 1–10 stored scale to the workbench's display scale.
• Repeat-work rate — derive: share of the expert's leads from repeat customers (customer seen on a prior completed lead).
• Vendor rating (net-new) — a rating of the expert by the vendor. Add the minimal field + capture (or derive from an existing vendor-side signal if Define finds one); aggregate per expert.
• Rework rate (net-new) — a rework signal on delivered work (e.g. a milestone/lead reopened or a rework flag); rate = reworked / completed. Add the minimal signal + capture.
• Wire the quality cards to the four real figures.

Acceptance criteria (rough)

• CSAT shows the expert's real average, correctly scaled.
• Repeat-work rate is derived from real repeat-customer leads.
• Vendor rating reflects a real vendor-supplied rating (or honest "—" when none).
• Rework rate reflects a real rework signal (or honest "—" when none).
• No hard-coded quality values remain.

Out of scope (this feature)

• The earnings half of the Performance section (feature 7).
• A full review/ratings subsystem beyond the minimal vendor-rating + rework capture needed here.

Notes for Define

• Vendor rating and rework are the net-new pieces — if either needs more than a field + minimal capture (e.g. a structured review flow, a rework lifecycle), spin it out via /feature decompose before building, per the epic flag.
• CSAT scale: the model validates 1–10; the mock shows 4.8 — confirm the display scale (likely /5).
• touches: packages/services/src/db/models (vendor-rating + rework signal), packages/services/src/db/services/matching/expert-metrics.ts (reuse CSAT), packages/services/src/server, apps/web/components/dashboard/expert/performance-section.tsx. </content> </invoke>

Spec: Expert quality metrics

• slug: expert-quality-metrics
• issue: #469
• personas: Expert, Vendor, SDM, CSM, Customer
• touches: packages/services/src/db/models/lead.ts, packages/services/src/db/models/expert-rating.ts, packages/services/src/db/models/index.ts, packages/services/src/db/services/matching/expert-metrics.ts, packages/services/src/server, apps/web/app/(app)/service-leads/[id], apps/web/app/(app)/expert/dashboard/page.tsx, apps/web/components/dashboard/expert/performance-section.tsx
• complexity: complex

Problem

The Quality card of the expert workbench's Performance section is hard-coded: csat, vendorRating, repeatWorkRate, and reworkRate all come from mock data (apps/web/lib/mock/expert.ts). An expert can't trust the feedback loop that's meant to "support self-improvement" when the numbers are fictional. This is the last of the expert-dashboard-data epic (8 of 8) and advances Build the Bridge / Q2 2026 Objective 3 — Validate Technical Infrastructure & Payout Flow: real quality signals are what let us validate that the matching and delivery flow produces trustworthy expert performance data. Coverage is mixed today — CSAT has a model and an avg-score helper and repeat-work is derivable from leads, but vendor rating and rework rate have no backing concept at all.

Proposed change

Compute all four quality figures from real data and wire the Quality card to them. Each figure shows an honest em-dash (—) when it has no data yet, never a fabricated number.

• CSAT — reuse the existing per-expert average-CSAT computation (the csat model + getMetricsForExpert in expert-metrics.ts). CSAT is stored on a 1–10 scale and the card displays / 5, so map the average to the /5 scale (divide by 2) and round to one decimal. CSAT links to an expert via its lead.expert (the csat model stores lead/customer/vendor, not expert directly), so the aggregate joins through leads.
• Repeat-work rate — derive from the expert's leads: a customer is "repeat" for the expert when they have more than one completed lead with that expert; rate = completed leads from repeat customers ÷ total completed leads, shown as a percentage. (A completed lead is the same definition already used by getMetricsForExpert: isActiveProject: false, bidPool: false, isBrdApproved: true.)
• Vendor rating (net-new, minimal capture) — add a small expert-rating model holding a vendor's 1–5 rating of an expert for a given lead (tenantId, vendor, expert, lead, score). Capture is a minimal 1–5 star control on the lead-detail page (apps/web/app/(app)/service-leads/[id]) shown to the vendor on a completed lead. The card figure is the average of all of that expert's vendor ratings.
• Rework rate (net-new, minimal capture) — add a minimal isReworked boolean to the lead model. Capture is a minimal "flag as reworked" control on the lead-detail page shown to SDM/CSM on a completed lead. Rate = the expert's completed leads flagged reworked ÷ the expert's completed leads, shown as a percentage.
• Expose all four figures from a single per-expert quality function in @sustentus/services (extending ExpertMetrics / the matching service, or a sibling quality function) and wire the Quality card in the expert dashboard to them, replacing the mock quality object.

Acceptance criteria

• CSAT shows the expert's real average from completed CSAT records, mapped to the /5 display scale, or — when the expert has no completed CSAT.
• Repeat-work rate is computed from the expert's real repeat-customer completed leads and shown as a percentage, or —/0% when the expert has no completed leads.
• A vendor can set a 1–5 rating of an expert on a completed lead from the lead-detail page, and that rating is persisted in the new expert-rating model.
• Vendor rating on the card reflects the real average of that expert's vendor ratings, or — when there are none.
• An SDM or CSM can flag a completed lead as reworked from the lead-detail page, persisting isReworked on the lead.
• Rework rate on the card reflects real reworked completed leads as a percentage, or —/0% when the expert has no completed leads.
• No hard-coded quality values remain in the expert dashboard's Quality card path.

Out of scope

• The earnings half of the Performance section (the expert-earnings-summary feature, 7 of 8).
• A full review/ratings or rework-lifecycle subsystem beyond the minimal expert-rating model + lead isReworked flag and their two capture controls (per the epic flag — if either grows, it splits out).
• Editing/deleting/moderating vendor ratings, multi-dimensional vendor reviews, free-text review bodies, notifications, or any vendor-rating/rework surfacing outside the lead-detail capture and the expert Quality card.
• Back-filling historical vendor ratings or rework flags.

Open questions

• none. (Sequencing note, not blocking: this feature depends on expert-workbench-foundation (1 of 8), which converts the expert dashboard from mock to real server-fetched data. The services, models, and capture controls here land independently; the final card wiring attaches to whatever data path the foundation establishes. If the foundation hasn't merged when Build runs, wire the card through an equivalent server fetch rather than reverting to mock.)

Build notes: expert-quality-metrics

• branch: feat/469-expert-quality-metrics
• commits:
  • feat: expert-quality-metrics — quality model + service layer
  • feat: expert-quality-metrics — lead-detail capture + wire dashboard card

What changed

Services (@sustentus/services)

• db/models/lead.ts — added a minimal isReworked boolean (default false) to the lead interface + schema (the rework signal).
• db/models/expert-rating.ts (new) — minimal expertRating model: a vendor's 1–5 score of an expert for a lead, tenant-scoped, unique per (tenant, lead, vendor). Registered in db/models/index.ts.
• db/services/matching/expert-quality.ts (new) — getExpertQualityMetrics(expertId, tenantId) returns the four real figures, each null when there's no data:
  • csat — average overallScore of completed CSATs joined to the expert through their completed leads (the csat model has no expert field), mapped to /5 via CsatService.toDisplayScore.
  • repeatWorkRate / reworkRate — derived over the expert's completed leads (isActiveProject:false, bidPool:false, isBrdApproved:true): repeat-customer share and isReworked share, as percentages.
  • vendorRating — average of the expert's expertRating scores.
  • Exported from db/services/matching/index.ts.
• db/services/expert-rating/{index,instance}.ts (new) — expertRatingService with findForLeadByVendor (preselect) and upsertForLead (derives the expert from the lead, upserts one rating per vendor+lead). Exported from db/services/index.ts.

Web (apps/web)

• app/(app)/service-leads/[id]/quality-actions.ts (new) — setExpertVendorRatingAction (vendor-only) and setLeadReworkAction (csm/sdm-only) using resolveActionContext + runActionBody, revalidating the lead page.
• components/service-leads/expert-vendor-rating/expert-vendor-rating.tsx (new) — 1–5 star control.
• components/service-leads/lead-rework-flag/lead-rework-flag.tsx (new) — flag/clear rework control.
• app/(app)/service-leads/[id]/page.tsx — renders the vendor-rating control (vendor, completed lead with an expert) and the rework control (csm/sdm, completed lead with an expert) in the right column.
• app/(app)/expert/dashboard/page.tsx — converted to a server component that fetches getExpertQualityMetrics for the signed-in expert and feeds the Quality card real values; the other (not-yet-built) sections still read mock. Honest "—" when quality can't be resolved.
• components/dashboard/expert/performance-section.tsx — Quality card now accepts nullable csat/vendorRating and renders "—" (no stars, no /5) when absent. No hard-coded quality values remain in the card path.

Acceptance criteria status

• CSAT shows the expert's real average, /5-scaled, or "—" — via getExpertQualityMetrics + CsatService.toDisplayScore, wired into the card.
• Repeat-work rate derived from real repeat-customer completed leads (percentage, or "—"/"0%").
• Vendor can set a 1–5 rating on a completed lead from the lead-detail page → persisted in expertRating.
• Vendor rating on the card = real average of the expert's vendor ratings, or "—".
• SDM/CSM can flag a completed lead as reworked from the lead-detail page → isReworked on lead.
• Rework rate on the card = real reworked-completed-lead share, or "—"/"0%".
• No hard-coded quality values remain in the expert dashboard's Quality card path.

Verify result

• Format · lint · typecheck · build run in CI + the Vercel preview, not here. No check is expected to fail.

Notes for review

• Sequencing: the dependency expert-workbench-foundation (1 of 8) hasn't merged, so the expert dashboard page was server-converted just for the Quality card (per the spec's sequencing note); the remaining sections deliberately stay on mock until the foundation lands.
• CSAT→expert join: the csat model stores lead/customer/vendor (not expert), so the aggregate joins through lead.expert over the expert's completed leads rather than the pre-existing getMetricsForExpert CSAT.find({ expert }), which matched nothing.
• Capture is intentionally minimal (one star control + one flag toggle) per the epic's "field + minimal capture, don't grow into a subsystem" flag.

Changelog — expert quality metrics

See real quality scores for every expert

Experts

• Your go-live workbench now shows your real quality scores — customer satisfaction, platform rating, repeat-work rate, and rework rate — instead of sample numbers.
• Each figure shows a clear "—" when there is not enough data yet, so you never see an invented score.

Vendors

• You can now rate an expert from one to five on a delivered lead, and your rating feeds that expert's platform rating.

Customer success and service delivery managers

• You can now flag a delivered lead that needed rework, which feeds the expert's rework rate.

Expert quality metrics are now real, not illustrative

Sustentus experts now see their true quality signals on the go-live workbench — customer satisfaction, platform rating, repeat-work rate, and rework rate — each computed from real delivery data instead of placeholder numbers. This advances Build the Bridge and our Q2 2026 Objective 3: Validate Technical Infrastructure & Payout Flow: trustworthy quality data is what lets us stand behind the experts we match and the deliveries we process end to end.

• All four quality metrics now read from live platform data, showing an honest "—" until real feedback exists — no invented numbers.
• Vendors can now rate an expert on a delivered lead, and delivery managers can flag work that needed rework — two quality signals we previously had no way to capture.
• It builds directly on the matching and delivery data already flowing through the platform, turning it into a feedback loop experts can trust.

Ship: expert-quality-metrics

• PR: #471 — https://github.com/sustentus/sustentus/pull/471
• branch: feat/469-expert-quality-metrics
• CI: format <pending> · lint <pending> · typecheck <pending> · preview build <pass — web + docs + all apps deployed green on the build commit>
• technical docs: no technical docs impact — the technical/packages/services page documents entrypoints/structure, which are unchanged (a new model + service function doesn't alter the documented altitude).
• business docs: updated business/feature-role-matrix/experts (view quality metrics), …/vendors (rate expert on a delivered lead), …/leads (flag delivery reworked) in this PR.
• integration: quality is filled into the merged expert-workbench-foundation's getWorkbench quality slot (foundation now live on main), so the whole workbench reads from one server fetch.
• release notes: both — investor draft in 03_ship/output/investor-update.md; changelog entry appended to apps/help/app/changelog/page.mdx.

Review summary

• Two independent review passes (correctness + typecheck/soundness) found no bugs. Verified: expert-id identity (lead.expert == expert user _id, since "expert" is a discriminator on the user collection), CSAT avg-then-/5 scaling, nullish (not falsy) handling so a real 0% rate isn't shown as "—", upsert filter matches the unique index, all export chains + nullable types propagate. — accepted.
• Non-blocking note: the expertRating partial-index $or mirrors the existing csat.ts pattern — not a regression.
• Merged main into the branch (the dependency expert-workbench-foundation #460 had landed). Integrated real quality into the foundation's expertWorkbenchService.getWorkbench quality slot (the slot it left for this feature) instead of a standalone page fetch; ExpertWorkbenchQuality made nullable. Conflicts in the dashboard page, changelog, and services barrel resolved.

Acceptance check (vs spec)

• CSAT shows the expert's real average, /5-scaled, or "—" — getExpertQualityMetrics averages completed CSATs (joined via lead.expert) through CsatService.toDisplayScore; card renders "—" when null.
• Repeat-work rate from real repeat-customer completed leads (%) or "—"/"0%".
• Vendor can set a 1–5 rating on a delivered lead → persisted in expertRating (setExpertVendorRatingAction + expertRatingService.upsertForLead).
• Vendor rating on the card = real average of the expert's vendor ratings, or "—".
• SDM/CSM can flag a delivered lead as reworked → isReworked on the lead (setLeadReworkAction).
• Rework rate on the card = real reworked-completed-lead share (%) or "—"/"0%".
• No hard-coded quality values remain in the expert dashboard's Quality card path (page now server-fetches real metrics).

Merge & deploy

• merged: not yet — awaiting explicit human merge approval (gate:merge-approved).
• deploy: pending merge.