stuntman / providers / telemetry

Telemetry

An OpenTelemetry receiver that joins your app's spans to the provider calls Stuntman answered.

Stands in for

  • OpenTelemetry Collector (OTLP)
  • Graylog (GELF)
  • StatsD and DogStatsD
  • An APM backend

Everything Telemetry gives you

  1. 01

    Traces, logs and metrics

    OTLP over HTTP (protobuf or JSON) and gRPC. GELF and StatsD land in the same logs and metrics.

  2. 02

    Spans meet provider calls

    Stuntman reads the trace ID on the way in, so a span to Azure Blob opens the call and the rule that answered it.

  3. 03

    Wait instead of sleeping

    wait_for_trace and wait_for_log return the first match after the agent acts in the app.

  4. 04

    Where the time went

    find_slow_spans lists spans over a threshold, provider spans first.

  5. 05

    Logs of one trace

    logs_of_trace gives every line the app wrote while serving a request; lines_around shows the context across services.

  6. 06

    Did the fix work?

    count_logs and compare_windows say whether an error stopped or a metric moved after a change.

  7. 07

    Why a metric jumped

    why_it_jumped finds the steps over twice the median and the breakdown value and Stuntman answer behind them.

  8. 08

    Calls that escaped

    real_provider_calls lists hosts the app still calls directly, from its own spans.

  9. 09

    Forward to a real APM

    Copy everything to a real backend; the API key is a Secrets reference, never a value.

The screens

Each number is a control or a reading on the real screen. Everything you can click there, your agent can do with a tool.

A trace joined to the provider calls

A trace joined to the provider calls: the Telemetry screen in Stuntman
1Touching a provider

Traces with spans that reached Stuntman, drawn in the provider's colour.

1 of 5
2Failed request

POST /orders: error 500 after 2 retries.

2 of 5
3Provoked spans

Two Azure Blob PUTs answered 503 by a Stuntman rule.

3 of 5
4The app gave up

No third attempt: the retry policy is visible in the span tree.

4 of 5
5Answered by

The rule that answered the span, and a link to the object in Files.

5 of 5
  1. Touching a provider

    Traces with spans that reached Stuntman, drawn in the provider's colour.

  2. Failed request

    POST /orders: error 500 after 2 retries.

  3. Provoked spans

    Two Azure Blob PUTs answered 503 by a Stuntman rule.

  4. The app gave up

    No third attempt: the retry policy is visible in the span tree.

  5. Answered by

    The rule that answered the span, and a link to the object in Files.

Logs with their trace

Logs with their trace: the Telemetry screen in Stuntman
1Level and service

Counts per level and service; filter with list_logs or count_logs.

1 of 5
2The error

The line the app wrote when the order failed.

2 of 5
3Retries

Warnings for each 503 the app got, with the attempt number.

3 of 5
4Trace id

Every line keeps the trace id the app sent, so it opens its trace.

4 of 5
5Stuntman's side

Which rule made the uploads fail and why the app gave up.

5 of 5
  1. Level and service

    Counts per level and service; filter with list_logs or count_logs.

  2. The error

    The line the app wrote when the order failed.

  3. Retries

    Warnings for each 503 the app got, with the attempt number.

  4. Trace id

    Every line keeps the trace id the app sent, so it opens its trace.

  5. Stuntman's side

    Which rule made the uploads fail and why the app gave up.

Metrics and why they jumped

Metrics and why they jumped: the Telemetry screen in Stuntman
1Semantic names

Metrics grouped by provider calls, the app, runtime and custom.

1 of 5
2Rule on

p95 jumps while a Stuntman rule was failing uploads.

2 of 5
35xx

Errors the app answered while the rule was on.

3 of 5
4Not Stuntman

A host the app still calls directly: a forgotten base URL.

4 of 5
5Why it jumped

The step, the breakdown value that drove it and what Stuntman answered.

5 of 5
  1. Semantic names

    Metrics grouped by provider calls, the app, runtime and custom.

  2. Rule on

    p95 jumps while a Stuntman rule was failing uploads.

  3. 5xx

    Errors the app answered while the rule was on.

  4. Not Stuntman

    A host the app still calls directly: a forgotten base URL.

  5. Why it jumped

    The step, the breakdown value that drove it and what Stuntman answered.

Break it on purpose

The failures you need to test, exactly when you ask: from the screen or by a tool call, each one shown on the timeline with who caused it.

  • Unauthorized

    The collector answers 401 or 403

    provoke_collector 401 | 403
  • Throttled

    The collector answers 429

    provoke_collector 429
  • Unavailable

    The collector answers 503

    provoke_collector 503
  • Refused

    Connections to the collector are refused

    provoke_collector refused
  • Slow collector

    Exports take longer: does the app block?

    provoke_collector slow
  • Half dropped

    Half of the batches are dropped

    provoke_collector drop_half
  • Signal off

    Traces, logs or metrics answered 200 and dropped

    set_signal on false

Why not the real sandbox

Your AI agent drives it

Claude Code, Codex or any MCP client connects to /mcp (or calls /api over plain HTTP), sets the stand-in up, breaks it on cue and reads what your app did.

you

After the order fails, find out why and whether the app really retried the upload.

wait_for_trace{"route": "POST /orders", "status": "error", "timeout": "30s"}
→ ok trace 4bf92f35…a1e0 · 842 ms · error 500 · 14 spans
find_slow_spans{"trace_id": "4bf92f35a1e0", "over_ms": 100}
→ ok Azure Blob PUT × 2 · 503 · rule "Contract uploads fail twice" · no third attempt
real_provider_calls{"since": "15m"}

MCP tools

  • telemetry_overviewServices, counts, refused batches, endpoints
  • list_tracesTraces by service, status, slowness or provider
  • get_traceThe span tree with provider trips
  • wait_for_traceWait for the first matching trace
  • find_slow_spansSpans over a threshold, provider spans first
  • logs_of_traceEvery line of one trace
  • count_logsCheck an error stopped after a fix
  • why_it_jumpedFind what drove a metric jump
  • provoke_collectorMake the collector fail
  • real_provider_callsHosts the app called directly

The same over HTTP

  • GET/api/telemetry/traces/wait
  • GET/api/telemetry/traces/{trace_id}
  • GET/api/telemetry/traces/{trace_id}/logs
  • GET/api/telemetry/metrics/{name}/why
  • POST/api/telemetry/collector/provoke
  • GET/api/apis/real-provider-calls