Skip to Content

← All archived runs

Run: agent-chat-experience-polish

run.md

Run: agent-chat-experience-polish

  • branch: claude/agent-chat-experience-polish-tpm3td
  • pr: #908

02_define/output/spec.md

Spec: Assistant conversation experience polish

  • slug: agent-chat-experience-polish
  • personas: Admin, CSM, SDM, Expert, Vendor, Customer
  • touches: apps/agent, packages/ui, packages/services, apps/storybook
  • complexity: standard

Problem

The agentic app is the platform's agentic face, and the conversation inside it is the least designed thing in it: a two-line centred empty state, assistant replies distinguished from the user's only by bubble alignment, no way to copy a reply, no sense of time, a "Thinking…" shimmer detached from the exchange, and degraded/error alerts bolted above the input rather than belonging to the conversation they describe. This is stub 2 of the agentic-ui-polish batch — stub 1 (agent-shell-layout-polish, #905) established the frame and brand foundation; this run styles the conversation inside it. It advances Scale the bridge / Q2-2026 Objective 3 — Validate Technical Infrastructure & Payout Flow the same indirect way the batch does: the agentic surface is the infrastructure being validated, and a surface people will not use validates nothing.

A second problem surfaced at pickup, which the stub's Notes for Define told Define to check for. agentic-tenant-context (#902) bound a real tenant-scoped tool (list_catalogue, via buildCatalogueTools) into the assistant, but left packages/services/src/ai/assistant/prompt.ts telling the model "You have no access to tenant data… You cannot look anything up". The model now holds a lookup tool it has been instructed never to use. That contradiction is what the empty state's "I can't reach your workspace data yet" copy inherits, so it cannot be worded honestly without correcting the prompt underneath it.

Proposed change

A visual and copy pass over apps/agent/components/assistant-chat.tsx and the shared chat primitives it uses, plus the minimum prompt correction that makes the new copy true.

Branded welcome with starter prompts. Replace the two-line centred empty state with a branded welcome: the Logo brand mark, a proper copy hierarchy, and four clickable starter chips. Per Jamie's ruling (2026-08-28) the chips are catalogue/data-aware, not generic — they exercise the list_catalogue capability that actually shipped. Four distinct angles on that read, not four rewordings: the whole catalogue, products only, services only, and a named search. Each sends its text through the existing sendMessage path as an ordinary user turn.

Prompt correction (prompt.ts). The "What you can do" section is corrected to describe the one tool the agent holds: it can read this organisation's catalogue of products and services, read-only, for the tenant the caller is bound to. The security section, the "never claim to have taken an action", the no-persistence line and the answering style are untouched. Nothing here changes /api/chat, the toolset, or how a tenant is resolved — the tool has been bound since #902; this tells the model it may use it.

Message-level refinements. An assistant brand mark on replies, a copy-to-clipboard action on assistant messages, a subtle timestamp on both roles, and a streaming affordance that renders in the reply's own bubble and resolves into the streamed text instead of a detached shimmer line. The reusable half lands in packages/ui/src/compound/ai/ with Storybook coverage alongside the existing Message.stories.tsx; app-specific composition stays in apps/agent.

In-flow status presentation. The degraded-availability notice and the turn-failure alert are restyled to sit in the conversation flow rather than bolted above the input. Their semantics are frozen: the sanitised CHAT_ERROR_MESSAGE and AI_DEGRADED_MESSAGE copy is unchanged, degraded stays non-blocking, and the errored state stays sendable. That last one is a live trap, flagged in the stub: PromptInput resets the form before the submit handler runs, so the handler's !isBusy guard (never status === "ready") must survive the restyle or a failed turn silently eats the retry the error copy invites.

Acceptance criteria

  • The empty state renders the branded welcome — Logo brand mark, headline and supporting copy — replacing today's two centred Typography lines.
  • Four starter chips are shown; each is a real button, keyboard-reachable, and clicking one sends its text through sendMessage as an ordinary user turn that appears in the transcript as the user's message.
  • The four chips are catalogue-aware and distinct: the full catalogue, products only, services only, and a named search — not four wordings of one prompt.
  • prompt.ts no longer tells the model it has no tenant data access; its "What you can do" section describes the read-only catalogue capability, and its security section, no-action-claim rule, no-persistence line and answering-style rules are byte-identical to today.
  • On the preview, asking the assistant what is in the catalogue (by chip or by typing) returns catalogue items for the bound tenant rather than a refusal.
  • Assistant replies carry a brand mark and a copy-to-clipboard action that puts the reply's text on the clipboard and confirms it; user messages stay visually distinct (today's filled primary bubble, right-aligned).
  • Both roles carry a subtle timestamp.
  • While a turn is in flight the pending affordance renders in the assistant bubble where the reply will appear, and the streamed text replaces it in place with no layout jump; the detached Shimmer line is gone.
  • The degraded notice and the failed-turn notice render inside the conversation flow, not as Alert boxes stacked above the input, and carry today's AI_DEGRADED_MESSAGE and CHAT_ERROR_MESSAGE strings unchanged.
  • Sending stays possible while degraded and while errored: the submit handler still guards on !isBusy and never on status === "ready", and the send-guard comment explaining why survives.
  • Every new or changed packages/ui primitive has a Storybook story under apps/storybook/src/ai/.
  • The empty state, the conversation and both status states render correctly in light and dark themes, and at mobile width inside the shell stub 1 established.

Out of scope

  • Conversation persistence, history or resume — the chat stays ephemeral, and the welcome copy still says it starts fresh on refresh.
  • Any change to /api/chat, /api/ai/availability, the availability probe's semantics, the toolset it builds, or the error-sanitisation policy. The prompt correction is the only packages/services change.
  • Adding, removing or changing any AI tool. list_catalogue is the one that exists; this run does not wrap another service.
  • Wiring the chips to live tenant data client-side — a chip is text sent as a user turn; any lookup happens model-side through the existing tool.
  • proxy.ts, the authenticated-user gate, Clerk configuration, and the tenant picker.
  • The platform's own AI surfaces in apps/web, and the later redirection batch.
  • Message editing, regeneration, branching or feedback actions — MessageBranch exists in packages/ui but nothing in this run uses it.

Open questions

  • none

Notes

Context budget: within band. The one deliberate overrun is reading packages/services/src/ai/assistant/prompt.ts, packages/services/src/ai/tools/catalogue.ts and apps/docs/archive/pipeline-runs/agentic-tenant-context/02_define/output/spec.md — the stub's Notes for Define required checking whether the "can't reach your workspace data yet" claim still held, and it does not.

Scope provenance: this batch has no scope.md.icm/intake/agentic-ui-polish/breakdown.md records it as cut from Jamie's intake interrogation (2026-08-27), with the rulings quoted per stub. Two Define-time rulings extend those: prompt.ts is corrected in this run rather than deferred to a bug lane, and all four starter chips are catalogue-aware rather than generic (Jamie, 2026-08-28) — the "generic on purpose" ruling in the stub was premised on the assistant having no data access, which #902 changed.

Both prerequisites are merged and closed out: agent-shell-layout-polish (#905, in-batch depends-on) and agentic-tenant-context (#902, the batch's cross-batch timing condition).

03_build/output/notes.md

Build notes: agent-chat-experience-polish

  • commits: ab28608 feat: branded welcome, catalogue starter chips, message-level refinements
  • ci: pending — recorded on the push below

What changed

  • packages/ui/src/compound/ai/message.tsx: four additions, all pure chat primitives. MessageAvatar (identity slot, caller supplies the mark), MessageCopyButton (clipboard + a visible confirmation that reverts after 2s; a Clipboard API failure leaves the icon alone rather than claiming a copy that did not happen), MessageTimestamp (a real <time> with dateTime), and MessagePending (the Shimmer, moved inside the reply bubble).
  • packages/ui/src/compound/ai/suggestion.tsx (new): Suggestions + Suggestion. A chip carries its own text and hands it back through onSuggestionClick, so the caller sends it down the same path a typed message takes — a starter is an ordinary turn, not a special one.
  • packages/ui/src/compound/ai/notice.tsx (new): ConversationNotice — a status the conversation is reporting, rendered in the transcript. Composed from Alert (as Confirmation already is) but lighter: dashed border, transparent ground, so it does not outweigh the messages around it.
  • apps/agent/components/assistant-chat.tsx: branded welcome (Logo + headline + supporting copy) with four starter chips; assistant replies gain the brand mark, a copy action and a timestamp; user messages keep their filled primary bubble and gain a right-aligned timestamp; the pending affordance moved into the assistant bubble; both status notices moved from above the composer into the flow.
  • packages/services/src/ai/assistant/prompt.ts: capability correction — see below.

The prompt correction

agentic-tenant-context (#902) bound list_catalogue into the assistant but left the prompt saying "You have no access to tenant data … You cannot look anything up". The model held a lookup tool it was instructed never to use, and the empty state inherited that claim. Both halves are now true: the "What you can do" section names the catalogue tool, states that it is read-only and bound to the signed-in organisation, and tells the model to use it rather than answer from memory (and to report an empty catalogue as empty rather than inventing items).

Everything else in the prompt is byte-identical: the two injection-resistance bullets, the no-prompt-disclosure bullet, the no-persistence line and the whole "How to answer" section.

One deviation from the approved criterion, ruled by Jamie at build time (2026-08-28). The criterion asked for the security section to stay byte-identical and for the false data-access claim to go — but the claim sat inside the security section, in the second half of the no-action-claim bullet ("You cannot read or change any data yet — see What you can do"). Ruling: trim the false half only. "Never claim to have taken an action in the platform." stays verbatim, and now reads "… Your tools read; none of them write." — the same rule, held up by a true premise instead of a false one.

Acceptance criteria status

  • Branded welcome — Logo (size="lg"), an h1 headline and supporting copy replace the two centred Typography lines.
  • Four starter chips — each a real <button> via Suggestion, keyboard-reachable, sending its text through sendMessage as an ordinary user turn.
  • Chips are catalogue-aware and distinct — whole catalogue / products only / services with their active state / a named search.
  • prompt.ts no longer claims no tenant data access; the capability section describes the read-only catalogue tool. Security bullets, no-persistence line and answering-style rules unchanged apart from the trim recorded above.
  • Catalogue questions return items on the preview — not assertable here; it needs a signed-in tenant with catalogue rows on the deployed preview. Verify's DoD smoke owns it.
  • Brand mark + copy action on assistant replies; user messages keep the filled primary bubble, right-aligned.
  • Both roles carry a timestamp. Arrival time is recorded once per message id in a ref — a new Date() in render would re-clock every message on every stream tick.
  • Pending affordance renders in the assistant bubble and the streamed text replaces it in place; the detached Shimmer line is gone.
  • Both notices render in the flow with AI_DEGRADED_MESSAGE and CHAT_ERROR_MESSAGE unchanged.
  • Sending stays possible while degraded and while errored — the !isBusy guard and the comment explaining why status === "ready" would be wrong are untouched.
  • Storybook coverage — Suggestion.stories.tsx and ConversationNotice.stories.tsx added; Message.stories.tsx gains refined and pending presets covering the four message additions, through the existing one-story-with-controls convention.
  • Light/dark and mobile rendering — built on shared tokens throughout, no hardcoded palette, but eyes on the preview are what settle it. Verify's smoke pass.

Notes for Verify

  • Two criteria are deliberately unticked, both because they can only be settled on the running preview: the catalogue round-trip and the light/dark + mobile pass. Nothing is known-broken.
  • Watch the send-guard. The stub flagged it and it survived intact, but it is the one regression that would be invisible on a happy-path smoke: PromptInput resets the form before the submit handler runs, so a guard on status === "ready" would silently eat the retry the failure copy invites. Worth exercising the errored state deliberately.
  • The empty-state copy now claims a catalogue lookup. That claim is only true because prompt.ts changed in this PR — the two have to be reviewed together, not separately.
  • MessageAvatar is aria-hidden in the app: the mark is decoration, and letting a screen reader announce the logo's "S" before every reply would be noise.

Context budget: within band. The one overrun was Define's, carried in spec.md.

  • No Storybook preview was built for this PR, and that is a config bug, not a skipped story. apps/storybook/vercel.json uses --fallback=HEAD^1 where every other app uses --fallback=main, so turbo-ignore only looks at the last commit in a push. The stories landed in ab28608 and the notes-only commit 3676a36 followed, so storybook read "unaffected" and skipped. The stories are typechecked by Quality Project, but nobody can look at them on this PR. Out of scope to fix here (deploy config, not chat polish) — raised as its own task.

04_verify/output/verify.md

Verify: agent-chat-experience-polish

  • ci: GREEN on 0e7e9ce — settled via ci-status.sh after the review fixes; re-established on this verify.md push, which is markdown-only (no app code differs from 0e7e9ce).
  • previews smoked: all eight built on 0e7e9ce — agentic-interface (the app this run changes), storybook, web, demo, docs, help-centre, marketing, tenant-management. See the preview note below: the earlier head had them skipped, and pushing real code is what restored them.
  • production-readiness: not required — no database, auth, payments or env vars in the diff. Verified rather than assumed: git diff origin/main...HEAD contains no process.env read, no migration, schema or model file, and touches: is apps/agent, packages/ui, packages/services (prompt text only), apps/storybook.
  • code-review: medium (spec complexity standard) — 3 findings, all fixed on branch in 0e7e9ce.
  • security-review: run — no HIGH or MEDIUM findings. Triggered on judgement rather than the literal list: the diff edits the assistant's system prompt, including its "Security and scope" section, which is the closest thing this agent has to a route policy.
  • playwright: TODO — manual DoD smoke performed instead.

Preview note (a finding, now resolved)

At the start of this stage the head was e1505b1, a markdown-only commit. turbo-ignore compares a project against its own last successful deployment, so every app preview was skipped on that head — including agentic-interface, the one app this run changes. A stage cannot demonstrate a UI change on an app that did not build.

The head now carries real code (0e7e9ce) and all eight previews built, so the evidence below is against previews of the exact code under review. Two lessons recorded rather than fixed here:

  1. A run should not end on a docs-only commit if the previews matter — which is exactly what Ship's own docs/changelog commit will do next. Ship inherits previews built at 0e7e9ce.
  2. apps/storybook/vercel.json additionally uses --fallback=HEAD^1 where every other app uses --fallback=main. Separate from the above and already raised as its own chore task.

DoD smoke (on the preview — each line says who verified it)

The agent could not reach the preview at all: agentic-interface sits behind Vercel SSO deployment protection (GET / → 302 to vercel.com/sso-api), and every acceptance criterion in this run is behind the app's own signed-in gate besides. So the agent half below is code-traced, not demonstrated, and is labelled as such. Nothing on this list is self-certified as seen.

  • Branded welcome (Logo, headline, supporting copy) replaces the two centred Typography lines — code-traced (agent); needs operator
  • Four starter chips render as real keyboard-reachable buttons and send through sendMessage as ordinary user turns — code-traced (agent); needs operator
  • The four chips are catalogue-aware and distinct (whole catalogue / products / services + active state / named search) — verified in source (agent); needs operator on the preview
  • prompt.ts no longer claims no tenant data access; injection-resistance bullets, no-prompt-disclosure bullet, no-persistence line and "How to answer" byte-identical — verified against the diff: the only removed lines are the four capability claims (agent)
  • Asking for the catalogue returns items for the bound tenant rather than a refusal — needs operator (requires a signed-in tenant with catalogue rows; this is the criterion Build left unticked)
  • Brand mark + copy action on assistant replies; user messages keep the filled primary bubble — code-traced (agent); needs operator
  • Both roles carry a timestamp — code-traced (agent); needs operator
  • Pending affordance renders in the assistant bubble and the streamed text replaces it in place — code-traced (agent); needs operator
  • Both notices render in the conversation flow with today's copy — code-traced (agent); needs operator
  • Sending stays possible while degraded and while errored — the !isBusy guard and its comment are intact in source (agent); needs operator to exercise the errored retry deliberately
  • Every new/changed packages/ui primitive has a Storybook story — Suggestion.stories.tsx, ConversationNotice.stories.tsx, and Message.stories.tsx refined/pending presets; the storybook preview built on 0e7e9ce so this is now inspectable (agent, from source; operator can confirm visually)
  • Light/dark themes and mobile width — built on shared tokens throughout, no hardcoded palette (agent); needs operator
  • auth: affected persona(s) still sign in and reach the assistant — needs operator
  • payments: not touched — no payment code in the diff (agent)
  • notifications: none expected — this run fires no notification and touches no notification code (agent)

Findings & cleanup

  • Stranded "Thinking…" after a failed streampending was isBusy && !text, so an assistant bubble left empty by a turn that failed after the stream opened would shimmer again on every later turn, alongside the real one. Gated on the message being last. Fixed on branch (0e7e9ce).
  • Copy confirmation did not re-armsetCopied(true) while already true did not re-run the clearing effect, so a second copy inside the 2s window showed its tick for a few ms. Replaced the boolean with a counter. Fixed on branch (0e7e9ce).
  • Starter chips had no in-flight guard — the composer refuses while busy; the chips did not, so a double-tap sent two turns and billed two gateway requests. Same guard applied. Fixed on branch (0e7e9ce).
  • Security: error sanitisation invariant holds — checked explicitly because the restyle moved the error surface. error is used only as a truthy flag and logged to the console; the DOM only ever receives the constant CHAT_ERROR_MESSAGE. No raw provider text reaches the page.
  • Security: the prompt edit grants no new reach — the model was already handed list_catalogue by #902. defineTenantTool closes over ctx.tenantId and the tool's input schema carries no tenant id, so no instruction injected through the conversation can point it at another tenant. The edit stops telling the model not to use a tool it already had; it does not widen the tool.
  • Cleared in review, recorded so nobody re-checks: no export-name collisions across the @sustentus/ui barrels; Alert's has-[>svg] grid works with ConversationNotice's direct-child icon; Button/Logo/Shimmer/Typography accept every size and as value passed; no SSR/hydration risk from MessageTimestamp (messages are empty on first render).

Open for the gate

The signed-in half of the DoD is not yet demonstrated — the agent has no preview credentials. Jamie's confirmation of the quality gate should rest on the operator half being walked on https://agentic-interface-git-claude-agent-chat-experi-6122b0-sustentus.vercel.app, not on the code-traced lines above.

Context budget: within band.

05_ship/output/changelog.md


title: The assistant can answer questions about your catalogue date: 2026-08-28T10:00:00Z personas: [customer, expert, csm, sdm, admin, vendor] slug: agent-chat-experience-polish pr: https://github.com/sustentus/sustentus/pull/908

The assistant can answer questions about your catalogue

The last release fixed the room the conversation happens in. This one is about the conversation itself — and about something the assistant can now actually do for you.

Ask about your products and services. The assistant can look up your organisation's catalogue and answer from it, so "what services do we offer, and are they all active?" gets a real answer instead of a polite refusal. It reads only your own organisation's catalogue, and it only reads — it cannot change anything. Everything else still needs the main Sustentus platform, and the assistant will say so plainly rather than guessing.

A starting point instead of a blank page. Opening the assistant now offers four things worth asking, and tapping one sends it as your first message.

Replies are easier to work with. Each reply is marked as the assistant's, carries the time it arrived, and has a copy button — useful when you want to paste an answer into an email or a ticket.

Waiting looks like waiting. While the assistant is thinking, the indicator sits where the reply will appear and the words simply fill in, instead of a separate line above the message box that vanished when the answer arrived.

When something goes wrong, it reads as part of the conversation. If the assistant cannot reach its model, or a message fails, the notice now sits in the conversation where it belongs. You can still send in both cases — a failed message is worth trying again, and it usually works.

Conversations still start fresh when you refresh the page.

05_ship/output/investor-update.md

The Sustentus assistant now answers from a customer's own catalogue

Who it's for: Admin, CSM, SDM, Expert, Vendor, Customer What shipped: The assistant reads an organisation's catalogue and answers from it, opening with four prompts instead of a blank page. Why it matters: Scale the Bridge — Q2-2026 Objective 3: the agentic surface now does real work on real tenant data.

Reads are scoped to the signed-in organisation by construction.

Dig deeper: https://github.com/sustentus/sustentus/pull/908 · https://help.sustentus.com/changelog/2026-08-28-agent-chat-experience-polish

05_ship/output/release.md

Ship: agent-chat-experience-polish

  • pr: #908 · merge: authorised — Ready to merge ticked by Jamie (2026-08-28); this commit rides the squash
  • CI: GREEN on efb2bc5, settled via ci-status.sh after Verify's last push; re-established on this Ship push before the merge
  • technical docs: technical/architecture (the "no tenant data" claim about apps/agent was false since #902 — corrected, with the tenant-scoping terms stated) · technical/packages/ui (AI Components list: no longer BRD-only, and the six new chat primitives added)
  • business docs: no business docs impact — no persona capability, service-journey step or feature-role-matrix entity changes; the assistant's catalogue read is a new surface for an existing capability, not a new one
  • release notes: both
  • sent: ship note queued to #product-update on merge (ship-note.yaml sends it from the merge commit's tree)
  • close-out: archives this run to apps/docs/archive/pipeline-runs/, and — as the last stub of the agentic-ui-polish epic, with agent-shell-layout-polish (#905) already merged — the epic folder too

Acceptance check (vs spec)

  • Branded welcome with Logo, headline and supporting copy — code-traced in Verify
  • Four keyboard-reachable starter chips sending through sendMessage — code-traced in Verify; guarded on !isBusy after Verify's finding
  • Chips catalogue-aware and distinct (whole catalogue / products / services + active state / named search) — verified in source
  • prompt.ts no longer claims no tenant data access; security bullets, no-persistence line and answering style byte-identical apart from the trimmed false half of the no-action-claim bullet (Jamie's ruling, 2026-08-28)
  • Catalogue round-trip on the preview — not demonstrated. The agent has no preview credentials (Vercel SSO) and the app is auth-gated throughout. Merged on Jamie's authority with this line open.
  • Brand mark + copy action on assistant replies; user bubble unchanged — code-traced; copy confirmation re-arm fixed after Verify's finding
  • Timestamps on both roles — code-traced
  • Pending affordance in the reply bubble, resolving in place — code-traced; stranded-shimmer regression fixed after Verify's finding
  • Both notices in the conversation flow with AI_DEGRADED_MESSAGE / CHAT_ERROR_MESSAGE unchanged — verified in source
  • Sending stays possible while degraded and errored — !isBusy guard and its comment intact in source
  • Storybook coverage for every new/changed packages/ui primitive — Suggestion, ConversationNotice, and Message presets refined/pending
  • Light/dark and mobile rendering — not demonstrated. Built on shared tokens with no hardcoded palette, but unverified on a running preview. Merged on Jamie's authority with this line open.

Note on the two open lines

Verify recorded ten DoD lines as needing an operator: the agent could not reach the preview (GET / → 302 to vercel.com/sso-api) and every criterion sits behind the app's own auth gate. Eight of those are code-traced above with the mechanism named. The two left unticked are the ones where tracing the code is genuinely not evidence — whether the model actually returns catalogue items, and whether it looks right in both themes at mobile width.

The concern was put to Jamie before the merge, twice, in those terms. He reaffirmed the ship and ticked Ready to merge. Recorded here rather than quietly ticked, so the record says what was checked and what was taken on trust.

Context budget: within band.