AI Gateway — billing and availability

Every agent on the platform reaches its model through the Vercel AI Gateway, addressed by gateway model string (anthropic/claude-haiku-4.5, anthropic/claude-sonnet-5) rather than a raw provider SDK. One account funds all of it, so that account running dry degrades every AI surface at once — the onboarding concierge and the assistant in the agentic app, which also holds the customer’s requirements (BRD) conversation.

This page is the runbook for that account: who owns it, how it is funded, what warns you before it runs dry, and how to prove live AI still works without spending a demo.

The arrangement

The credential, and where it has to be set

AI_GATEWAY_API_KEY is declared in turbo.json → globalEnv, but that only governs what turbo passes into build tasks under strict env mode — it says nothing about runtime. A serverless function reads its own Vercel project’s runtime environment, which turbo.json does not populate.

So the key must be set per project, for Production and Preview, on each project that reaches the gateway — today agentic-interface (the availability probe and the assistant) and web (the BRD chat and the onboarding concierge). Two projects, two independent environment sets. “One account funds all of it” is true operationally and unenforced technically: if the two projects’ keys ever point at different accounts, the balance the probe reports is not the balance the chats spend.

The probe cannot check this for you. With no key the SDK falls back to the deployment’s OIDC identity; if that is absent too the call throws, which resolves to unknown — and unknown renders nothing. In exactly the misconfiguration this page exists to prevent, the signal is silent and the only trace is a console.error naming the error class. Read the env var directly (vercel env ls --scope sustentus, or the project’s settings) rather than inferring it from a quiet UI.

Where the balance lives

The remaining credit balance is on the Vercel team’s AI Gateway page, under the sustentus team. The same figure is what the platform reads programmatically (gateway.getCredits()) — so the dashboard and the in-product signal agree provided the key is set and points at this account. If a project falls back to OIDC (see above) the probe reads the deployment identity’s balance instead, and can report ok while every turn is refused.

Topping up

Credits are bought from that page by the account owner. Nothing in the codebase performs, schedules or authorises a top-up — it is an account action, on purpose.

Why there is no spend cap

A cap would mean the product changes behaviour as spend rises, and that was ruled out: the alert is for a human to react to, and product behaviour never responds to spend. The alert threshold is the operator’s signal; the availability probe below is the user’s signal, and it reports “cannot serve”, never “spending a lot”. The two are not the same instrument and should not be conflated.

That decision still stands — nothing enforces a per-tenant budget, and no request is ever refused for spending. What has changed is that the threshold is no longer the only instrument. Run accounting prices every metered turn from the gateway’s reported usage against a versioned price table, and two screens attribute it:

• /spend in the agentic app — a tenant administrator’s own spend, by the agent that spent it, month by month.
• /ai-spend in the console — the same figures across the estate, per tenant; staff see every tenant, an invited partner sees only the tenants assigned to them.

Both are attribution and early warning, not billing: they are labelled metered usage, they state the population, period and basis behind every figure, and they say when a turn ran on a model the price table cannot price rather than reporting an incomplete total as complete. The gateway’s own records stay authoritative, which is why the console surfaces recent generation ids to reconcile a sample by hand. The project budget remains the only hard backstop.

What the product does when the gateway declines

Two surfaces, two behaviours — none of which ever shows the raw provider error. That error text carries provider names, account tiers and a billing top-up link, and it has previously reached a tenant’s setup screen.

In both, the underlying error goes to the browser console and the server log.

The availability signal

GET /api/ai/availability in the agentic app answers a signed-in caller with one of three verdicts, derived from the credit balance:

• ok — balance above zero.
• degraded — balance at or below zero: the account is out of prepaid credit. Whether the gateway then refuses a turn depends on the plan, so the warning says the assistant may not reply and sending stays enabled. Confirm what this gateway does at a zero balance on pay-as-you-go — if it keeps serving, this verdict is a false alarm shown to every signed-in user, and the threshold should move rather than the copy.
• unknown — the balance could not be read or the probe itself failed. Nothing is shown. A check that cannot run must not tell every user the assistant is down.

The response carries the verdict and nothing else — never the balance, the account tier, or gateway error text. Results are cached for 60 seconds, so opening the conversation repeatedly does not spend a gateway round-trip per mount.

Smoke-testing live AI

Canned mode covers demos: on the demo tenant the agents replay a recorded transcript, so a walkthrough never touches the gateway and never costs anything. That is also why a passing demo is not evidence that live AI works — the two paths do not overlap.

To prove the live path:

1. Use a non-demo tenant. Canned mode is resolved from the tenant record, so any other tenant is already live.
2. Send one turn in the agentic assistant and one in the onboarding concierge. One turn each is enough — the failure this guards against is total, not intermittent.
3. Confirm a real reply streams back. A friendly “unavailable” message means the gateway declined: check the balance before assuming a code fault.

Do this on the Vercel preview before merging anything that touches an agent, and once on production after it deploys.

Proving the degraded path

Now that the account is funded, the degraded branch cannot be reached by using the product normally — which makes it the branch most likely to rot unnoticed. Do not verify it by draining the account. Exercise it deliberately instead:

• In the browser — with the conversation open, respond to /api/ai/availability with {"status":"degraded"} (request interception in devtools) and reload. The warning must appear above the composer, and the composer must still send.
• In the unit tests — apps/agent/lib/ai-availability.test.ts pins the balance-to-verdict mapping, including that an unreadable balance and a failed probe both resolve to unknown rather than degraded.

When AI is down

1. Read the balance on the Vercel AI Gateway page.
2. If it is at or below zero, top up. Every surface recovers on its own — no deploy, no restart. The server-side probe cache expires after 60 seconds; the conversation asks once when it mounts, so a tab that is already open keeps showing the warning until it is reloaded.
3. If the balance is healthy and AI is still failing, it is not a billing fault: check the Vercel runtime logs for the failing route (/api/chat in the agentic app, /api/onboarding/chat in the platform).