Observability

The Observability service records a trace of every inference request — how long it took, what it cost, which model and provider served it, and which pipeline stages ran — so you can see exactly what your application is doing and where time and credits go.

You will find Observability under Services > Observability.

Turning Observability on

Use the Enabled switch in the top right of the page. It takes effect the moment you flip it — there is no separate save step. While it is off, no traces are recorded and nothing is metered. Turning it on starts recording from the next request; it never changes how your requests are served.

How recording works

When the service is on, each request to the inference API produces one trace with a span for every stage of the pipeline that ran — guard, memory, RAG, PII firewall, routing and inference, and PII restore. Each span is timed individually, so a trace is a true breakdown of where the request spent its time.

Recording happens after the response is returned, on a background queue, so observability never adds latency to the call your user is waiting on — traces appear in the explorer a moment after the response. Failed requests are traced too: a guard block, a PII block, an out-of-credits error, or a provider error each produce a trace stamped with the matching status, so you can see failures as clearly as successes.

The Traces tab

The Traces tab lists every recorded request, newest first. Each row shows the model, provider, status, latency, token count, cost in credits, and when it ran. You can filter by model, provider, status, and time range, and sort by any column.

Click a row to open a drawer with the full trace: its metadata and a span waterfall — one bar per stage, sized by how long that stage took, with failed stages shown in red. The waterfall makes it obvious whether time went into retrieval, the firewall, or the model itself.

The Metrics tab

The Metrics tab aggregates your traces over a window you choose — last 24 hours, last 7 days, or last 30 days. It shows:

• Headline figures — total requests, error rate, total spend, and p50 / p95 latency for the window.
• Request volume over time — how traffic rose and fell across the window.
• Latency over time — p50 and p95 total_latency_ms per bucket, so you can spot slow periods.
• By model and by provider — which models and providers carried the most traffic.

The window switches the whole tab; reads are free, so you can slice the data as often as you like.

Metering and retention

Each recorded trace is a single, metered event (observability.trace) and appears in your usage. Reading traces and dashboards is always free. If a request is not recorded — because the service was off, or because sampling skipped it — it is never charged.

Traces are kept for a fixed retention window and pruned automatically after it, the same way audit logs are. High-volume tenants can record a fraction of requests through a sample rate, which still meters only the traces that are actually recorded.

Good to know

Observability is metered per recorded trace. With the service on, every served request produces one billable trace event unless sampling skips it. Reads are free.

It never slows your requests. Traces are written on a background queue after the response is sent; if recording fails, your request is unaffected.

Trace metadata only. Spans record stage names, timings, and status — never your prompts, completions, or the content of retrieved context.