stuntman / providers / ai models
AI models
The model APIs your app calls, answered without a model: rules shape the reply, failures come on cue, spend is counted.
Stands in for
- OpenAI (chat completions, Responses, embeddings, models)
- Azure OpenAI (deployments and v1)
- Anthropic Messages (count_tokens, streaming)
- Google Gemini (generateContent, embedContent, countTokens)
- OpenRouter
- DeepSeek
Everything AI models gives you
- 01
Six APIs, your SDK
Change only the SDK's base URL. Any key is accepted and never sent anywhere.
- 02
Rules shape answers
First match wins on provider, API, model prefix, the last user message, the system prompt or a tool offered.
- 03
Tool calls on cue
The model calls your tool first with the arguments you set, then answers the text once the result comes back.
- 04
JSON and long answers
Answer a JSON object, pad to about N tokens, or stop by length as max_tokens would.
- 05
Streamed or not
Each provider's own response and error shapes, streamed or in one piece.
- 06
Every call kept
Prompt, answer, model, tokens in and out, cached tokens, cost and why it got that answer.
- 07
Spend you can read
ai_spend per provider; prices per million tokens are editable with set_ai_price.
- 08
Stable embeddings
Embeddings are deterministic: the same text always gives the same vector.
The screens
Each number is a control or a reading on the real screen. Everything you can click there, your agent can do with a tool.
Every model call, with why

OpenAI, Azure OpenAI, Anthropic, Gemini, OpenRouter and DeepSeek, each with its calls.
Any key is accepted; it is never sent anywhere.
The rule that matched, the default answer, or a failure someone forced.
System, user and the tools the app offered, as the app sent them.
Tokens in and out, cached tokens and the cost from your prices.
- Six providers
OpenAI, Azure OpenAI, Anthropic, Gemini, OpenRouter and DeepSeek, each with its calls.
- Point the SDK here
Any key is accepted; it is never sent anywhere.
- Why this answer
The rule that matched, the default answer, or a failure someone forced.
- The prompt
System, user and the tools the app offered, as the app sent them.
- Tokens and cost
Tokens in and out, cached tokens and the cost from your prices.
Rules and failures

When the message contains refund: call lookup_order, then answer the text.
A system prompt that asks to classify gets a JSON object.
Name, condition, answer text, and an optional tool with its arguments.
429 with Retry-After, 500, 529, filter, length, bad_json, slow and cut.
Bound the failure to the next N calls; kind=none clears it.
- A rule with a tool call
When the message contains refund: call lookup_order, then answer the text.
- JSON on a system prompt
A system prompt that asks to classify gets a JSON object.
- Add a rule
Name, condition, answer text, and an optional tool with its arguments.
- Failures on the next calls
429 with Retry-After, 500, 529, filter, length, bad_json, slow and cut.
- How many
Bound the failure to the next N calls; kind=none clears it.
Break it on purpose
The failures you need to test, exactly when you ask: from the screen or by a tool call, each one shown on the timeline with who caused it.
Rate limit429 with Retry-After, in the provider's error shape
force_ai_failure kind=429Server error500 from the provider
force_ai_failure kind=500Overloaded529 overloaded
force_ai_failure kind=529Content filteredThe answer is blocked by the content filter
force_ai_failure kind=filterCut by max_tokensThe answer stops with a length finish reason
force_ai_failure kind=lengthBroken tool argumentsA tool call whose JSON arguments do not parse
force_ai_failure kind=bad_jsonSlow model40 ms per token
force_ai_failure kind=slowStream cutThe stream stops halfway
force_ai_failure kind=cut
Why not the real sandbox
No tokens billed
Test suites and agents call the models all day; the spend counter counts, nobody pays.
Deterministic answers
The same prompt gets the same reply, so tests that depend on a model stop flaking.
Failures on demand
A 529, a filtered answer or a cut stream on the next call, not when the provider has a bad day.
Prompts you can inspect
Every prompt the app sent, with the tools it offered and the tokens it cost.
Your AI agent drives it
Claude Code, Codex or any MCP client connects to /mcp (or calls /api over plain HTTP), sets the stand-in up, breaks it on cue and reads what your app did.
Make our support bot retry when Anthropic is overloaded and still call lookup_order for refunds.
MCP tools
list_ai_callsCalls with provider, model, tokens, cost and whyget_ai_callOne call with the whole request and responsewait_for_ai_callWaits until the app calls a modellist_ai_rulesRules, the default answer and pricesset_ai_ruleAdds or changes a ruledelete_ai_ruleRemoves a ruleforce_ai_failureThe next calls fail on purposeai_spendCalls, tokens and cost per providerset_ai_pricePrice per million tokens, and the default answer
The same over HTTP
- GET
/api/ai/calls - GET
/api/ai/calls/wait - PUT
/api/ai/rules - POST
/api/ai/force - GET
/api/ai/spend
- With no rule the answer is "This is a simulated answer from Stuntman."; set_ai_price default_text changes it.
- Rules and prices live in ai/config.yaml; the screen, the API and agents edit the same file.