A learning demo: an auto insurance agent for a made-up New Jersey carrier. It quotes new customers and services existing ones. Stack: Claude Sonnet 4.5, Vercel AI SDK, Upstash Redis, Next.js.
Try a quote with a bad VIN, or log in as an existing customer. The middle pane is the live record, which is what the evals score.
Hi, I'm Ava, Pass^k Mutual's AI agent. This is a demo: the rates, policies, and people are all made up. Are you looking for a new auto quote, or help with a policy you already have?
Pick an example above, or type your own request.
Record (what gets scored)
The quote or policy record appears here as the agent works. Policy details only show up after the customer is verified.
Trace
No tool calls yet.
Record (what gets scored)
The quote or policy record appears here as the agent works. Policy details only show up after the customer is verified.
The quoting logic (VIN, license and date-of-birth checks, simulated MVR, CLUE and prior-insurance reports, underwriting rules, rating) is copied by a sync script from a voice quote line I'm building, so both price the same driver the same way. Mid-term changes re-rate the policy and prorate the difference.
12 tasks, 3 trials each, measured Oct 15, 2026. A simulated customer (claude-haiku-4-5) with a hidden goal talks to the agent (claude-sonnet-4-5). A trial passes only if the final record is right. Telling a customer they're covered from a quote fails the trial.
pass^3 is the chance that 3 fresh trials of a task all pass (unbiased estimator). Aggregate pass^1 is 1.000. The score covers only these custom tasks.
All rates, policies, people and driving records are synthetic. VINs decode against the public NHTSA database. Nothing here binds coverage. Sessions expire after a day. See also Pass^1 and passk.