tenant-notification-emailrun.md02_define/output/spec.mdtenant-notification-centre gave the console's own people somewhere to find out what happened —
but it is still a place you go. The people most affected by a tenant event are the ones who will
never go there: a tenant's own admins do not sign into this console and never will, yet their
workspace being suspended or closed is something they must be told about, and a partner can go
days without opening the app while a tenant of theirs is provisioned or moved. Email is the only
channel that reaches all three audiences without asking any of them to look.
This is the last stub in the tenant-management-app batch and the console's first outbound
surface to people outside it. It advances Scale the bridge / Establish product-market fit
with vendor partners (2026-Q2, Objective 1): the partner programme only works if a partner
administering tenants on a vendor's behalf hears about those tenants without being told to check,
and a vendor whose workspace changes hears it from us rather than from silence.
Every event that raises a notification in the centre also sends email, to three audiences, each with its own event set and its own wording [Q-14]. The centre is unchanged; this is delivery laid on top of it.
The event set is the centre's, not a second list. CONSOLE_NOTIFIABLE_ACTIONS in
packages/services/src/db/services/console-notification/index.ts is the one place that decides
what is notifiable, and email reads it. The staff and partner audiences additionally reuse the
centre's own two predicates — consoleActivityScopeFilter for who can see an event, and the
actor exclusion — so the two channels cannot disagree about who hears what.
The per-audience matrix. Seven actions are notifiable today:
| Action | Tenant's own admins | Assigned partner | Sustentus staff |
|---|---|---|---|
tenant.created |
✅ | ✅ | ✅ |
tenant.suspended |
✅ | ✅ | ✅ |
tenant.reactivated |
✅ | ✅ | ✅ |
tenant.deleted |
✅ | ✅ | ✅ |
tenant.recovered |
✅ | ✅ | ✅ |
tenant.removed |
✅ (see below) | — | ✅ |
tenant.user_role_changed |
— | ✅ | ✅ |
tenant.removed is absent because that trail entry carries no tenant by design (the
record has just been purged), so no partner scope can match it — the same reason it does not
appear in a partner's centre.Recipients.
org:admin, read via
the same readTenantMembers path the tenant detail view uses. A member with no email address
is a recorded failure, not a silent drop.tenant.partnerId. A tenant with no partner assigned
simply has no partner recipient, and that is not an error.sustentus
(readConsoleRoster), plus SYSTEM_ADMIN_EMAIL when it is set, deduplicated by address.
The roster is derived from Clerk invitations and cannot see someone granted console access
directly on an existing account — the bootstrapped admin especially — so the env address closes
that gap without a new mechanism or a new variable.Nobody is emailed about their own action: the actor is excluded by Clerk user id for the staff and partner audiences, exactly as the centre excludes them.
Permanent removal is handled ahead of time. removeTenantPermanently deletes the Clerk
organisation before it purges the tenant, so by the time the tenant.removed entry exists there
is no membership left to read. The tenant's admins are therefore resolved before
deleteTenantOrganization runs and carried through the removal, and mailed once the removal has
actually succeeded. If the removal fails or races a recovery, nothing is sent.
Wording. Three distinct templates per audience, not one template with a switch:
support@sustentus.com, also
set as the mail's Reply-To. No link into the console, no console vocabulary ("tenant",
"partner", "estate"), no mention of the actor.Console links are built from CONSOLE_APP_URL and the existing notificationHref rule, so an
email and a centre row lead to the same place. Unset CONSOLE_APP_URL degrades the mail to
linkless rather than mailing a link to nowhere.
Sending never affects the action. The administrative write and its trail entry commit first;
the fan-out is dispatched after the response via after() from next/server, and no failure in it
propagates to the caller. A plan the operator changed stays changed even if the mail does not go
out — which is exactly why the failure has to be written down.
Failures are recorded against the event. A new append-only ConsoleEmailDeliveryLog
collection stores failures only, one row per intended recipient, naming the activity entry it
belongs to, the action, the audience, the recipient address, the reason and a truncated detail.
It mirrors notification-delivery-log's shape and rules — no soft delete, no edit, a 30-day TTL
index for bounding, autoIndex off with a matching migration — but is its own collection because
the platform log requires a tenantId that tenant.removed cannot supply, and its recipients are
platform user rows, which console people are not. The failure taxonomy is
not-configured · transport-error · recipient-missing · recipients-unresolvable (the Clerk
read for an audience failed). Deliberate non-delivery — no partner assigned, an audience with an
empty event set for this action, the actor excluded — is not a failure and is never recorded.
Failures are visible to staff. A staff-only /notifications/delivery page lists recent
failures — time, tenant, action, audience, recipient, reason — each linking to the tenant where
one still exists, and the console dashboard carries a staff-only alert when anything failed in the
trailing 24 hours, following apps/web's NotificationHealthAlert pattern: silent at zero, and
calling out a missing RESEND_API_KEY distinctly from a per-send fault. Partners see neither
surface.
CONSOLE_NOTIFIABLE_ACTIONS; adding a notifiable action to the centre requires a decision in the email matrix before the code compiles, and no second list of actions exists.support@sustentus.com as the contact and sets it as Reply-To.CONSOLE_APP_URL unset the mail sends linkless rather than with a broken link.ConsoleEmailDeliveryLog against the activity entry that caused it, with the audience, recipient and reason; deliberate non-delivery is never recorded as a failure./notifications/delivery lists recent failures to staff and is refused to partners; the dashboard alert appears only when the trailing 24 hours contain a failure and distinguishes a missing API key from a per-send fault.RESEND_API_KEY unset, nothing is sent, every intended recipient is recorded as not-configured, and the dashboard says so — no exception reaches an operator.ConsoleEmailDeliveryLog's indexes are created by a migration matching the schema declarations, with autoIndex off, and the collection is bounded by a 30-day TTL.tenant.plan_changed event. The console has no plan-change action to notify on: it was dropped from tenant-plan-visibility because Clerk's backend Billing API has no plan-change write, and CONSOLE_NOTIFIABLE_ACTIONS records the same gap. When the action arrives it is one more row in the matrix above, not a rebuild. The stub's "its plan changed" is therefore unbuildable this run.tenant.user_role_changed, and notifying the partner of tenant.removed — both are deliberate absences in the matrix above, not omissions.notification-delivery-log, the platform's own failure log. It stays exactly as it is.03_build/output/notes.mdfeat: tenant-notification-email — email the three audiences on every console tenant eventc1f7d3c — Quality Project (format/lint/typecheck/test), Review diff against CONVENTIONS.md, both migration checks, and the tenant-management preview buildpackages/services — the part that needs no network calldb/models/console-email-delivery-log.ts — new append-only, failures-only collection.
Its own rather than a row in notification-delivery-log: that model requires a tenantId the
tenant-less tenant.removed event cannot supply, its type is the platform's notification
union, and its recipients are user rows, which org-less console people do not have. 30-day
TTL, autoIndex off, one createdAt read index — the staff read has no tenant clause to lead
with and must reach the tenant-less rows.db/migrations/1788520000000-console-email-delivery-log-indexes.ts — creates both, collation
named explicitly, up re-runnable, down symmetric.db/services/console-email-delivery-log/ — the read side: recentFailures and
failureSummary, both declared crossTenant because they genuinely are.notifications/console/audience.ts — the per-audience matrix, a total Record over the action
union so a new trail action cannot compile until somebody has decided who hears about it.notifications/console/copy.ts — the two vocabularies, pure. Tenant admins get "your
workspace", no actor, no link; the partner and staff get the tenant named and a way back in.notifications/console/emails/ — two templates. The tenant-admin one deliberately does not
build on emails-shared/layout.tsx: that layout ends in a "View in app" button and its reader
has no app to view it in.notifications/console/send.ts — notifyConsoleTenantEvent. Applies the actor exclusion, dedupes
by address, sends, records every failure. Never rejects.notifications/console/record-failure.ts — the log write, reusing describeDeliveryError and
createFirstOccurrenceGuard from the platform recorder rather than copying them.db/services/console-activity/index.ts — record() now returns the entry id, so a failed send
is attributable to the event that caused it rather than to a matching timestamp.db/services/tenant/index.ts — ConsoleDeletedTenant carries partnerId (same projection,
one more field), so recovering a tenant can email the partner who administers it.demo-reset/policy.ts + tenant-purge/index.ts — the new collection classified in both
registries. Kept on a demo reset like the trail; purged with the tenant on permanent removal.apps/console — the part that is all Clerklib/console-event-email.ts — resolves the three audiences. Every read is wrapped: a Clerk
failure becomes null for that audience, which the fan-out records as unresolvable rather than
raising. Staff = the invitation roster plus SYSTEM_ADMIN_EMAIL.lib/console-activity.ts — the fan-out hangs off the two existing record helpers, which every
administrative action already goes through, dispatched via after().tenants/{new,[tenantId],deleted}/actions.ts — pass the tenant's organisation and partner from
reads that already happened; resolveTarget and the lifecycle read now carry both.tenants/deleted/actions.ts — reads the tenant's admins before deleteTenantOrganization,
mails them only once the removal has actually succeeded.notifications/delivery/page.tsx + components/email-delivery-alert.tsx + proxy.ts — the
staff-only failure surface, gated twice (proxy matcher and requireStaff()).CONSOLE_EMAIL_MATRIX,
resolved in console-event-email.ts, sent in send.ts.CONSOLE_NOTIFIABLE_ACTIONS and no second list exists — the
matrix is keyed by the action union (compile-enforced) and agreement.test.ts asserts the
two agree in both directions.tenant.partnerId, the centre's own scope key.prepareAudience, once, for every
audience at the same time.support@sustentus.com as contact and Reply-To — asserted in copy.test.ts.notificationHref, the centre's own rule; unset
CONSOLE_APP_URL sends linkless.readPartnerRecipients returns [], which is not a failure.deleteTenantOrganization and
the record call is after the success branch.after() dispatch, and
notifyConsoleTenantEvent never rejects./notifications/delivery is staff-only and the dashboard alert distinguishes a missing key
from a per-send fault.RESEND_API_KEY unset records not-configured for the intended recipients and surfaces on
the dashboard, with no exception reaching an operator.autoIndex off, 30-day TTL.after() is load-bearing, not a style choice. A bare floating promise in a serverless
function races the runtime freezing once the response flushes — cancelled work writes nothing to
the failure log, which is the exact silence this feature exists to end. Worth confirming on the
preview that a lifecycle change still redirects at the same speed.not-configured guard is per-process, per-audience. On the preview, where
RESEND_API_KEY may be unset, expect one row per audience rather than one per recipient. That
is deliberate (it mirrors the platform recorder) but reads as under-reporting if you are counting.consoleActivityService.record() changed its return type from void to the entry id. Every
existing caller ignores it, so nothing else moved, but it is the one change in this diff that
touches shipped behaviour rather than adding to it.agreement.test.ts and copy.test.ts were written from the
acceptance criteria and are for the Quality workflow's Run tests step to judge.db-migrate.yaml
applies it on merge.04_verify/output/verify.mdci-status.sh settled green on 2956d82, the last commit carrying application
code, and again on each documentation-only commit after it, the last of which is the head handed
to Ship. Named this way deliberately: a stage that ends by writing its own record cannot name its
final SHA inside that record without chasing itself. Ship re-establishes the verdict on the head
it merges anyway — .icm/scripts/ci-status.sh tenant-notification-email.2956d82; see the preview note below for why that is the head's code. Every other Vercel project
is recorded as skipped for this diff, never quoted as a green preview.Review diff against CONVENTIONS.md) enabled and triaged; the
production-readiness pass carried the deep read for complexity: complexci-status.sh reported Vercel – tenant-management as skipped on head a12c62a, and
tenant-management is the app this entire feature lives in. That mismatch is a finding the Verify
contract asks for explicitly, so it is recorded rather than glossed:
The skip was correct and not a gap. a12c62a changed exactly one file —
03_build/output/notes.md — so Vercel's ignore step found nothing to rebuild for that project.
The console preview had built at c1f7d3c, the commit carrying all the app code, and
git diff c1f7d3c a12c62a is that single markdown file.
Resolved, and stated precisely rather than inherited. The fixes below are real code, so
Vercel – tenant-management rebuilt on the push carrying them: the console preview above is built
for 2956d82. The head handed to Ship is 495e518, which adds only this verify record —
git diff 2956d82 495e518 is 04_verify/output/verify.md and nothing else, so the deployed
console preview is byte-identical in app code to the head.
This is structural, not an oversight: a markdown-only push always skips every app preview, so the last commit of any stage that ends by writing its record can never itself carry a console build. The check that matters is the one made here — that the diff between the built commit and the head contains no application code.
Blocking, fixed:
tenant.created mail to the tenant's admins could never send. createTenantOrganization
creates the Clerk organisation empty by design (no createdBy — console people are org-less),
and the first admin then arrives as a pending invitation, not a membership.
readTenantAdminRecipients read memberships only, so at tenant.created the audience resolved
to [] — which the fan-out treats as deliberate non-delivery: no mail, and no failure row
either. The flagship "your workspace is ready" copy was unreachable, and the same gap silently
covered every later event for an admin who had not yet accepted.
Fixed: the tenant-admin audience is now the union of org:admin memberships and pending
org:admin invitations. An invited person is the right recipient regardless — they were given
the workspace, and the address is one a human chose.
A redundant createdAt index. The model declared { createdAt: -1 } and
{ createdAt: 1 } (TTL). Mongo accepts both because the directions differ, but a single-field
index is traversable either way, so the TTL index already served the newest-first list and the
$gte window — the second b-tree was pure write amplification on an append-only collection.
Exactly the fault the model's own tenantId comment refuses. Cheaper to fix now than after the
migration auto-applies on merge.
Fixed: one index, createdAt_ttl, in both the schema and the migration.
Also fixed:
The actor exclusion missed SYSTEM_ADMIN_EMAIL. The fallback staff recipient is a bare
address with no Clerk id, and the exclusion matched on id only — so the bootstrapped admin, the
person the fallback exists for and the one most likely to be performing these actions in early
production, would have been emailed about every action they themselves took. Straight breach of
an acceptance criterion.
Fixed: exclusion now matches id or address, extracted as the pure isActorRecipient and
covered by six new unit tests written from the criterion.
after() itself could throw out of the record helpers. The try/catch was inside the
callback, so a throw from after() — which happens when called outside a request scope —
would have propagated into an administrative action, breaking the invariant the module states
as absolute. Latent today (every caller is a server action), live for the first cron or route
handler. Fixed: the after(...) call is wrapped too.
One person who is both assigned partner and roster staff got two byte-identical emails. The
dedupe Set was per-audience. Fixed: hoisted across audiences, and audiences are now
prepared sequentially so matrix order — not Clerk read timing — decides which copy wins.
/notifications/delivery was unreachable at zero failures. Its only link was the dashboard
alert, which renders null when nothing has failed — so staff could reach the page to diagnose
breakage but never to confirm health. Fixed: staff-only nav entry, beside Access.
A comment claimed one pass where the code makes two counts. Fixed: the comment now says
what the code does and why (tenantPlugin does not hook aggregate, so $facet would mean
hand-writing the cross-tenant escape instead of declaring it).
Recorded, not fixed here:
docs-sync and changelog-entry both apply and Ship must run them.apps/console Vercel project — RESEND_API_KEY,
RESEND_EMAIL_FROM, SYSTEM_ADMIN_EMAIL. All four reads are catalogued in turbo.json
globalEnv ✅, but apps/console is a separate Vercel project and these may today be set only
on web. Operator check below. The degradation is deliberate and now visible (not-configured
rows + dashboard alert) rather than silent — but on first production deploy an unset key means
every console email fails and staff see a red banner.Run against the diff, focused on the new route policy, the new collection, recipient resolution and the outbound content path. Nothing reached the HIGH/MEDIUM confidence bar. What was checked and why it holds:
/notifications/delivery is gated twice, and both locks are real: proxy.ts matches it in
isStaffRoute and redirects a non-staff marker (read live from Clerk, so revocation bites on the
next request), and the page calls requireStaff() — the inner lock a server action could not
bypass. /notifications itself stays viewer-level; only the one path underneath is staff-gated.crossTenant: true
with a documented reason, and both are reachable only from staff-gated surfaces. The dashboard
skips the read entirely for a partner rather than merely hiding the result, so a partner never
causes an estate-wide count.dangerouslySetInnerHTML, and the CTA URL is built from CONSOLE_APP_URL plus the centre's own
notificationHref — our origin, no user-supplied host or protocol.Agent-verified (code paths traced in the diff; no preview credentials or inbox):
CONSOLE_EMAIL_MATRIX is a total map over the
action union (compile-enforced) and agreement.test.ts asserts both directions (agent)isActorRecipient, six unit tests from the criterion (agent)support@sustentus.com as contact + Reply-To — copy.test.ts (agent)readPartnerRecipients returns [], which is not a failure reason (agent)deleteTenantOrganization, and mails only
on the success branch (agent)/notifications/delivery refuses partners — proxy matcher + requireStaff() (agent)up is re-runnable and down is symmetric (agent)Operator-demonstrated — NOT YET PERFORMED. These need the console preview and an inbox:
/notifications/delivery is reachable from the new nav entry and refuses a partner (operator)not-configured (operator)RESEND_API_KEY, RESEND_EMAIL_FROM and SYSTEM_ADMIN_EMAIL are set on the
apps/console Vercel project, and that the from-address is a verified Resend sender
(operator)05_ship/output/changelog.mdThe notification centre tells you what happened in the console — when you are in the console. Now the same events reach you when you are not, and reach the people who never sign in to it at all.
A workspace's own administrators are told what happened to it. They have no console account, so
until now nothing reached them: a workspace was created, suspended or closed and the people who run
it found out from whoever told them. They now get a plain message about their own workspace — no
console links, no internal wording, and support@sustentus.com to reply to.
Partners and the Sustentus team are told as operators, in the wording the console already uses, with a link straight to the workspace concerned. It lands on the same page the bell's notification would have taken you to.
The same seven events, the same audiences. Creating, suspending, reactivating, deleting, recovering and permanently removing a workspace, and changing somebody's role in one. Email and the notification centre are driven by one list, so the two can never tell you different things.
You are still never told about your own actions — including when the address is the one configured for the account you are signed in as.
A change never fails because an email did. Suspending a workspace succeeds whether or not the message goes out. If a message cannot be sent, the reason is recorded rather than lost: the Sustentus team sees a warning on the console dashboard when anything has failed in the last day, and a page listing exactly which messages did not arrive and why.
05_ship/output/investor-update.mdWho it's for: Tenant administrators, partners, the Sustentus team What shipped: Creating, suspending, closing or reopening a tenant now emails its administrators, its partner and staff — each in their own wording. Why it matters: Refine the Bridge: notification timing. The people affected stop learning from whoever thought to tell them.
Dig deeper: https://github.com/sustentus/sustentus/pull/901 · https://help.sustentus.com/changelog/2026-08-27-tenant-notification-email
05_ship/output/release.md3f6dacc before this push, settled via ci-status.sh; re-established on the head
that actually merges, after this push, per the contract's step 4.2.technical/applications/console — the /notifications/delivery route, a new
Email notification section (audience matrix, the two vocabularies, recipient resolution, the
failure log), the dashboard's staff-only delivery alert, and the provider env vars on the
tenant-management Vercel project. The centre's "It sends nothing" bullet was false as of this
change and is corrected rather than left to rot. technical/packages/services — the console
path is separate from the platform's per-resource notify path, with its own log; the source-tree
comment updated.business/platform-overview — the managed-lifecycle bullet now says the people
a lifecycle change affects are emailed about it.ship-note.yaml fires on the merge and emails
05_ship/output/investor-update.md to #product-update. Its Dig deeper line carries the real
PR and changelog URLs, no placeholder, because the merge is what sends it.close-out.sh tenant-notification-email archives this run to
apps/docs/archive/pipeline-runs/. .icm/intake/tenant-management-app/ still holds live stubs,
so the epic is not archived with it.All thirteen criteria are ticked on the PR and evidenced in
04_verify/output/verify.md; seven were traced through the diff by the agent and the rest rest on
the operator smoke recorded there.
CONSOLE_EMAIL_MATRIX, agreement.test.tsCONSOLE_NOTIFIABLE_ACTIONS; no second list — compile-enforced
total map, agreement asserted both directionsisActorRecipient on id or address, six unit testssupport@sustentus.com as contact and Reply-To — copy.test.tsCONSOLE_APP_URL is unset — notificationHref + readConsoleOriginafter()
wrapped, every path resolvesConsoleEmailDeliveryLog; deliberate non-delivery is not/notifications/delivery is staff-only and refused to partners — proxy matcher +
requireStaff(); the dashboard alert distinguishes a missing key from per-send faultsRESEND_API_KEY unset nothing sends, every recipient is recorded not-configured, and
no exception reaches an operatorautoIndex off, 30-day TTLRESEND_API_KEY, RESEND_EMAIL_FROM and SYSTEM_ADMIN_EMAIL on the tenant-management
Vercel project, with a verified Resend sender. Unset is a visible degradation, not a silent one,
but it is still every console email failing on first deploy.Context budget: within the Inputs table.