Investigate

Issues tab

Every issue across your workspace's runs, grouped so that the same failure in fifty traces is one row with a run count, not fifty cards. Each group carries a severity, a kind, who found it, whether it was confirmed against the spans, and an on-demand fix recommendation grounded in the runs it appeared in.

What the tab shows

Open Issues in the dashboard. The table lists one row per issue group inside the selected time window, worst severity and most runs first. Each row shows the issue's title and a one-line preview, its severity, its kind, how many runs it appeared in, how many of those runs confirmed it, and when it was last seen. Clicking a row opens the group detail in place; the open group lives in the URL, so a link to it can be shared.

A group is keyed on a fingerprint. For a signal detected in the spans, the fingerprint is the normalized error line of the span that failed, so the same tool failing the same way always lands in the same group. For an issue the investigator wrote up in prose, the issue is first classified into a kind and a short, entity-free pattern ("company update failed and was not retried"), and the group keys on kind plus pattern, so two runs that failed the same way on different records still share a row.

Groups are workspace-wide

Grouping ignores the trace list's filters and pagination. The Issues tab reads the whole workspace inside the time window, so a rare issue still appears even if none of its traces are on the current page of the trace list.

Where issues come from

Three writers file occurrences. The Found by column and filter tell them apart.

SignalDeterministic checks over the spans of every completed run, at no model cost: a tool call that returned an error, a check (tests, lint, typecheck, build) that did not pass, a subagent that stopped early, the same call repeated verbatim, a credential echoed into the trace. Filed by a background job as soon as the run completes.
AuditThe scheduled sweep's one-call audit of a run digest, for traces no cheap detector flagged. Filed without span ids, because the digest carries none; a verification pass then reads the spans and keeps only what they support.
InvestigatorThe judge's confirmed issue list from a run's verdict: what it established actually cost the run something. When the investigator judges a run the audit already filed on, its issues replace the audit's rows for that run.

Signals are deliberately stricter than the candidates the investigator sees. A tool error is kept only on a hard marker such as an explicit tool failure, a non-zero exit, a traceback or stderr block, an error class, or an HTTP 4xx/5xx status. A successful command whose output merely mentions the word "failed" does not file anything. Repeated calls collapse to one occurrence per run.

Issue kinds

Every occurrence has a kind from one shared vocabulary. The first five kinds are visible to code, so signals can file them. The rest describe what the run did wrong rather than what a tool printed, so only a judge can file them.

KindMeansFiled by
Tool errorA tool call returned an error, was denied, or timed out.signal, judge
Failed checkA check the run set out to perform (tests, lint, typecheck, build) did not pass or was never recorded.signal, judge
Subagent stoppedA delegated task hit a step cap, a timeout, or a deadline before finishing.signal, judge
Repeated callThe same tool call with the same arguments was issued three or more times.signal, judge
Credential exposedA credential-shaped token was echoed into span content.signal, judge
Change not persistedA write, update, or side effect the run attempted did not take effect and was not recovered.judge
Wrong resultThe run produced, chose, or acted on incorrect data or a wrong decision.judge
Skipped stepA required step, validation, or check was not performed.judge
Overclaimed completionThe final report or status claims more than the evidence supports.judge
Wasted workRetries, loops, or redundant actions burned budget without changing the outcome.judge
Slow stepA step exceeded its time budget.judge
OtherAn issue the investigator confirmed that fits no other kind.judge

Severity is low, medium, or high. A group shows the worst severity among its occurrences, and the list sorts by severity before run count.

Confirmed vs unconfirmed

An occurrence is confirmed when a model read the spans in full and found that they show both the failure and its consequence. The investigator's issues arrive confirmed, because the verdict is written from the spans. Audit claims are written from a digest, so each one goes through a separate verification pass before it is kept: one read-only loop per trace over all of that trace's claims (up to eight), with the cited spans located by code before the model's first turn.

ConfirmedThe span text shows the failure and what it cost. The row is stored with the verifier's quoted evidence line and the real span ids it rests on.
DismissedThe span text directly contradicts the claim. The row is deleted; nothing reaches the tab.
UnclearNeither. The row is stored exactly as the audit filed it, unconfirmed. A transport failure or a refusal also lands here, so verification can never delete a real issue by accident.

The verifier runs on a fixed model (Claude Opus 5.5) regardless of the workspace's judge setting. The model is chosen by measurement against independently judged production claims: a wrong dismissal removes a real issue from the tab, so the model must almost never make one, and among those that qualify the one that decides the most claims wins. OpenAI's GPT-6 family was measured too and left far more real issues unconfirmed. A signal group earns its confirmed count differently: a judge issue joins it when the investigator named that signal kind and cited the span carrying it. A signal group with zero confirmed runs is still a real span-level failure, only one no judge has yet tied to a run outcome.

Filters

The toolbar sits in the dashboard top bar, in the same slot the trace list uses. Search matches the issue title and preview. The Filter menu offers:

Time windowThe dashboard's date range; only occurrences inside it count, and a group's run count is the count inside the window
VariantThe prompt version or agent label the failing runs carried
Severitylow, medium, or high
KindOne of the kinds above
Found bySignal, Audit, or Investigator; counts only that writer's occurrences
Confirmed onlyGroups confirmed against the spans in at least one run
Show dismissedInclude groups you dismissed that have not recurred

The variant, kind, and source selectors always list every value present in the window, whatever the other filters say, so a selector can never restrict itself into an empty menu.

Group detail

A group opens like a ticket in an issue tracker: a readable main column and a properties rail.

  • Title and framing. The issue's one-line summary, the verbatim error text, and what its kind means.
  • Analysis and likely cause. A group-level explanation, generated the first time a member opens the group: one structured judge call over the group's occurrences, quoting up to eight of them (confirmed first, then newest) and the cited span content from three sample traces. Cached for the workspace; viewers see the cached one or a note that a member opening the group will generate it. The API can force a fresh call.
  • Suggested fixes. Numbered concrete changes, each tagged with where it lands: the prompt, a tool, application code, or infrastructure.
  • Occurrences. One row per occurrence, newest run first: time, trace, variant, the run's duration and cost, and who found it. Clicking a row opens the trace at the failing span for a signal, or at the verdict's Issues card for an investigator issue.
  • Properties. Severity, kind, runs, investigator status, first and last seen, project, variant, and the dismiss action.

Dismiss and restore

Dismiss hides a group from the list. It is not a permanent mute: the moment a newer occurrence lands, the group resurfaces on its own, because a failure that recurs after you dismissed it is exactly the one you want to hear about. Show dismissed lists dismissed groups that have not recurred, and Restore brings one back. Dismissing requires the owner or member role; viewers can read everything else.

Digests & health

The Issues tab is also the source for two parts of the scheduled-sweep digest:

Confirmed issuesThe digest's ranked issue list is the window's confirmed issue groups, ordered by severity and then by how many runs confirmed them. Unverified signals are not listed there; they collapse to a single count with a link back to this tab.
Health scoreThe 0-100 agent health score combines the run-failure rate over the window's runs, the confirmed-issue rate over the runs the sweep audited (confirmed issues can only come from runs it examined, so the share is over that sample, not the whole window), and investigated high-severity findings. Unconfirmed occurrences do not move it. The score is printed with its sample (how many runs were audited out of the window, and the monitoring scope when one narrows the sweep) and is flagged low-sample under 20 audited runs.

A confirmed issue is therefore never "nothing to say": a window with one is enough for a digest to send even when every fast detector was quiet.

API

The same three endpoints back the tab. They accept an API key or a logged-in session. Reads are no-spend, so viewers keep access; dismissing and generating a fix need the owner or member role. from and to are required ISO timestamps bounding a window of at most 366 days, and unknown query parameter names are rejected with a 400.

From a coding agent such as Claude Code, the MCP server exposes the same data as list_issues and get_issue. Ask it things like "what issues does prompt-v3 have?" or "which issues are new in prompt-v3 compared to prompt-v2?": list_issues filters by variant, kind and severity, and with compareVariant labels each issue as new, shared, or gone between two variants. To compare versions on bad behavior too, compare_variants puts 2-5 variants side by side: failure and failed-verdict rates, how often each behavior label appears, anomaly findings, new and gone issues, and average cost, tokens, and duration.

GET/api/errors?from=&to=key / session

The grouped list for the window: { groups, facets }. Optional filters: variant, severity (low | medium | high), kind, source (a comma list of signal, audit, judge), confirmed=1 (groups confirmed in at least one run), dismissed=1 (include dismissed groups). facets lists the variants, kinds, and sources present in the window, ignoring the other filters.

GETPATCH/api/errors/{fingerprint}?from=&to=key / session (PATCH: member)

GET: { group, occurrences, fix, runs } - the group's aggregate (null when the window holds no occurrences), its occurrences newest first (capped at 500), the cached fix recommendation if one exists, and each occurrence run's duration and cost. PATCH { dismissed: boolean } dismisses or restores the group and returns { dismissedAt }.

POST/api/errors/{fingerprint}/fix?from=&to=[&force=1]key / member

Return the cached fix recommendation, or generate one with a single structured judge call grounded in the window's occurrences and sample span content: { fix: { summary, likelyCause, fixes[{ title, detail, target }], model, generatedAt, sampleCount }, cached }. force=1 regenerates. 404 when the window has no occurrences, 502 when generation fails, 429 on a rate limit. Billed under the interactive judge quota.