Scenario Catalog
Scenarios live in packages/gecx-chat/src/testing/scenarios.ts and are reused by the showcase app and tests.
| Scenario | Purpose | Expected behavior |
|---|---|---|
welcome | First-run greeting | Streams welcome text and suggestion chips. |
order-status | Support flow for an order lookup | Emits tool call, tool result, shipping text, citations, and chips. |
invoice-lookup | Account support flow | Exercises structured lookup and citation rendering. |
commerce-recommendation | Shopping assistant flow | Emits product carousel and add-to-cart suggestion chips. |
add-to-cart | Approval-gated cart mutation | Emits an add_to_cart tool call that requires approval before visible cart state changes. |
unknown-tool | Fail-closed tool behavior | Emits an unregistered tool call and surfaces a failed tool-call part. |
invalid-tool-input | Schema validation behavior | Emits a registered tool call with invalid input and surfaces a failed tool-call part. |
rich-content-sampler | Renderer contract sampler | Exercises markdown, citation, chips, custom payload, and diagnostics. |
a2ui-return-flow | Generative UI return workflow | Streams A2UI v0.9 frames into an a2ui-surface, renders a return workflow, and logs an action envelope. |
handoff | Live-agent transfer simulation | Moves through requested, queued, connected, and completed handoff states. |
file-upload | Upload preview flow | Prompts the showcase upload UI. Selected files use chat.attachFile; the mock transport exercises the real transport.upload? contract end to end (validation → bytes uploaded → file part appended on the next user message). Progress events are emitted by the transport itself, not stubbed by the UI. |
error-lab | Error handling flow | Emits structured error parts for diagnostics. |
angry-customer | Sentiment escalation demo | Drives the SignalEscalator from a frustration-rule rising past threshold to a requestTransfer call. |
gecx eval scenarios
The 22 reference gecx eval scenarios live in apps/showcase/scenarios/ (mix of .scenario.ts and .scenario.yaml). They cover welcome flows, order status, invoice lookup, commerce recommendation, apply refund, file upload, handoff for billing, invalid tool input, latency budget, helpfulness baseline, hallucination guard, no-handoff resolution, rich-content sampler, server-side refund, tool-call accuracy, unknown tool, welcome tone, and more.
Run them with:
pnpm eval:showcase
# or
pnpm gecx eval ./apps/showcase/scenarios \
--baseline ./apps/showcase/baseline.eval.json \
--fail-on-regress
See Evaluation and Eval CLI.
Showcase Routes
| Route | Scenario focus |
|---|---|
/ | Overview and architecture |
/wow-tour | Guided PRD proof path across runtime, tools, diagnostics, and handoff |
/agent-builder | Agent-ready integration packet generator |
/vibe-lab | Agent recipe preview, message-part proof, and scaffold packet |
/components | A2UI catalog gallery — 20 installable surface presets |
/support | Order support, tools, citations, chips |
/commerce | Product recommendations and cart state |
/rich-content | Message part rendering |
/generative-ui | A2UI frame stream, generated surface rendering, frame inspector, and action log |
/agent-graph | Multi-specialist routing with live inspector |
/sentiment-demo | Sentiment / intent signals and escalation |
/voice | Voice composer, firstAudioMs overlay |
/computer-use | Sandboxed agent browsing |
/dashboards | Pre-built analytics widgets over seeded events |
/tools | Client tool approval and execution |
/handoff | Human handoff and upload |
/diagnostics | Trace timeline and debug bundle |
/analytics | Product journey metrics, analytics events, CSAT, and resolution tracking |
/integration-wizard | Setup snippets |
Verification
Run scenario unit and default showcase E2E coverage from the repo root:
pnpm test
pnpm e2e
Run the Coolaid demo E2E coverage when changing apps/agi-coolaid-stand:
pnpm e2e:coolaid
Coolaid Demo Scenarios
The Coolaid Stand app uses its own local scenario catalog in apps/agi-coolaid-stand/src/lib/coolaidScenarios.ts.
| Scenario | Purpose | Expected behavior |
|---|---|---|
coolaid-welcome | Branded concierge greeting | Streams welcome text and recommendation chips. |
coolaid-focus | Product recommendation | Streams recommendation text, product carousel rich content, citation, and chips. |
coolaid-refresh-picker | Branded generated UI | Streams an A2UI flavor tuner with sliders and an action routed into the cart approval flow. |
coolaid-add-cart | Cart mutation | Emits an add_coolaid_case client tool call that requires approval before cart state changes. |
coolaid-order | Order lookup | Emits a lookup_coolaid_order client tool call and order-status text. |
Relevant files:
packages/gecx-chat/src/testing/scenarios.tsapps/showcase/src/lib/demoState.tsxapps/showcase/tests/navigation.spec.tsapps/showcase/tests/support-gecx-chat.spec.tsapps/showcase/tests/commerce.spec.tsapps/showcase/tests/generative-ui.spec.tsapps/showcase/tests/tools.spec.tsapps/agi-coolaid-stand/src/lib/coolaidScenarios.tsapps/agi-coolaid-stand/tests/a2ui.spec.tsapps/agi-coolaid-stand/tests/coolaid-demo.spec.ts
Source:
docs/reference/scenarios.md