Reference

REST API

Everything Kalmia exposes over HTTP. Ingestion and read endpoints authenticate with your workspace API key; dashboard endpoints (evals, members, scans) authenticate with your logged-in session. All endpoints are served from your Kalmia instance at https://www.kalmia.dev.

Authentication

There are two authentication modes, and each endpoint accepts one or both:

API keyAuthorization: Bearer kal_live_… — ingestion, read, and analysis
SessionLogged-in workspace membership — dashboard endpoints

API keys are workspace-scoped and shown once at creation (see API keys). Every trace-scoped route re-verifies the trace belongs to the resolving workspace, so a key can never trigger paid analysis on another workspace's trace. A workspace you don't belong to returns 404, never 403.

Capture is SDK-only

The legacy LLM-proxy capture endpoints (the OpenAI-compatible chat-completions path and the Anthropic-compatible messages path under /api/v1) are disabled and return 410 Gone with an endpoint_gone error body. Capture traces with the SDK or an OTel exporter instead.

OpenAPI spec

The key-authenticated API is described by a public OpenAPI 3.1 document at a stable URL, with no login and no password gate, so you can generate a client, load it into a request tool, or hand it to a coding agent instead of probing:

curl https://www.kalmia.dev/api/v1/openapi.json

Unknown query parameters are rejected

Every read endpoint validates parameter names, not just values. A request like ?since=7d or ?search=timeout gets a 400 whose body lists unknownParams and supportedParams. A filter is never silently ignored, so an unfiltered page can never masquerade as a filtered one. A path under /api that no endpoint owns answers 404 with the same JSON shape (error, requestId), never an HTML page.

Ingestion & read

Authenticate with your workspace API key.

POST/api/otel/v1/tracesAPI key

Ingest OpenTelemetry spans (OTLP/HTTP, protobuf or JSON). Maps GenAI + OpenInference attributes. ≤5 MB and ≤200 distinct traces per request, ≤10k spans per trace; exporter-split traces are merged by span id.

POST/api/v1/traces/ingestAPI key

Ingest pre-built trace objects. Single trace or array (≤100 per request). Returns { ingested, traces, rejected, warnings }.

POST/api/v1/traces/{id}/feedbackAPI key

Attach or update end-customer feedback for a trace: { rating: 'up' | 'down', comment }. Provided fields overwrite the stored state, omitted fields keep it; no history. Accepted even while the trace is still ingesting. See Capture → Customer feedback.

POST/api/v1/replay/ingestAPI key (write-only allowed)

Push a browser session recording (rrweb events) captured by the SDK's recordBrowser; the SDK builds the body. ≤25 MB per request (413 above); a malformed body is 400. Distilled into evidence for the investigator and, for a watchable capture, stored for the in-app player. Session replay is currently switched off in Kalmia, so nothing is linked until it returns; see Capture → Session replay.

GET/api/v1/tracesAPI key (full)

List captured traces for the key's workspace, newest-arrival first. SQL-side filters: ?provider=, ?status=, and ?from=/?to= — an inclusive ISO window on ingested_at (server arrival time, also the sort and cursor axis), not on the caller-supplied created. ?status= takes health values (error | ok, derived from whether the run itself failed: its root span errored or it died on its last span; recovered mid-run errors do not count) or lifecycle values (in_progress | completed | stalled — the value items report in their status field, so items matched by ?status=error still show their lifecycle status). Eval traces are excluded unless ?includeEval=true. Paginated: ?limit= (default 100, max 500) and ?cursor=; the response carries nextCursor (null on the last page). count is the size of this page, not a workspace total — use /summary for totals. status, from, to, limit, and cursor are validated and unknown parameter names are rejected — a bad filter gets 400, never a silent ignore. Returns slim headers with span counts. Write-only keys get 403.

GET/api/v1/traces/summaryAPI key (full)

One aggregate over a time window, computed SQL-side: traceCount, erroredTraceCount (runs whose root span errored or that died on their last span; recovered mid-run errors do not count), retriedTraceCount, errorRate (erroredTraceCount / traceCount), costUsdSum with pricedCount, the cost split (llmCostUsdSum, toolCostUsdSum, toolCostByService — e.g. how much Exa cost this week) and its basis (toolCostReportedUsdSum = stated by the provider, toolCostRateCardUsdSum = estimated at the published rate, toolCostBasisByService per key), duration p50/p95, LLM latency p95 with llmLatencySampleCount (how many traces it rests on; null p95 + 0 samples means LLM durations are not being recorded), usage (totalTokensSum and turnsSum with tokenSampleCount / turnSampleCount, the traces each rests on), and per-score averages. ?from=/?to= required (ISO, half-open window on ingested_at, ≤92 days); optional ?model=, repeated ?variants=. Returns { window, current, prior, priorStatus, priorWindow } — prior is the equal-length window just before, for period-over-period deltas; when its roll-up cannot be computed (priorStatus: unavailable) it is null, never zeros. Eval traces are always excluded. Answers fleet-wide questions in one call instead of paging the workspace.

GET/api/v1/traces/{id}API key (full)

Fetch one trace in full: every span with complete input/output content, tool arguments and results, turn structure, and metrics — the same preprocessed shape the dashboard's trace view renders. 404 for an unknown id. Write-only keys get 403.

GET/api/traces/analyticskey / session

Time-bucketed per-variant trend series: latency p50/p95, throughput, tokens, cost, error rate, scores. ?from=/?to= required (ISO, ≤92-day window); optional ?model=, repeated ?variants=. Siblings: /facets (distinct models/variants in the window; takes only ?from=/?to=, a ?model= or ?variants= filter is a 400), /summary (window + prior-window KPIs), /behavior-rate (per-variant failed-verdict rate over time, one SQL aggregation over the whole window; window.traceCount says how many traces it rests on).

GETPOSTDELETE/api/annotationskey / session / member

Read (?traceId= or ?traceIds=a,b), create ({ traceId, label, color?, note? }), or delete ({ traceId, annotationId }) behavior annotations on traces. Reads admit viewers and keys; writes need owner/member or a full-scope key.

GETPOSTPUTDELETE/api/annotations/behaviorskey / session / member

Manage behavior definitions ({ label, description?, critical? }). DELETE also removes the label's annotations and 409s while the behavior is attached to eval cases. Reads admit viewers and keys; writes need owner/member or a full-scope key.

GET/api/annotations/labelskey / session

All unique annotation labels in the workspace (autocomplete source).

Read traces

# Newest error traces, 25 per page
curl "https://www.kalmia.dev/api/v1/traces?status=error&limit=25" \
-H "Authorization: Bearer kal_live_your_api_key"
# → { "traces": [{ "id", "created", "ingested_at", "status", "metadata", "spanCount" }], "count", "nextCursor" }
# Pass nextCursor back as ?cursor= to fetch the next (older) page.
# Full content of a single trace
curl "https://www.kalmia.dev/api/v1/traces/trace-1" \
-H "Authorization: Bearer kal_live_your_api_key"
# → { "trace": { "spans": [...], "metrics": {...}, "fullInput", "fullOutput", ... } }
# Last 7 days in one call: counts, errors, cost, latency, plus the prior week
curl "https://www.kalmia.dev/api/v1/traces/summary?from=2026-08-25T00:00:00Z&to=2026-09-01T00:00:00Z" \
-H "Authorization: Bearer kal_live_your_api_key"
# → { "window", "current": { "traceCount", "erroredTraceCount", "errorRate", "costUsdSum", "pricedCount", "durationP95", ... }, "prior", "priorWindow" }
# A guessed filter name fails loudly instead of returning the full listing
curl "https://www.kalmia.dev/api/v1/traces?since=7d" \
-H "Authorization: Bearer kal_live_your_api_key"
# → 400 { "error": "Unknown query parameter: "since". Supported query parameters: provider, status, from, to, limit, cursor, includeEval.", "unknownParams": ["since"], "supportedParams": [...] }

Ingest a trace

curl -X POST https://www.kalmia.dev/api/v1/traces/ingest \
-H "Content-Type: application/json" \
-H "Authorization: Bearer kal_live_your_api_key" \
-d '{
"id": "trace-1",
"input": [{"role": "user", "content": "Hello"}],
"output": "Hi there!",
"metadata": {"provider": "openai", "model": "gpt-4o"},
"spans": [{
"id": "span-1", "span_id": "span-1", "root_span_id": "span-1",
"name": "openai chat",
"span_attributes": {"type": "llm"},
"metrics": {"prompt_tokens": 10, "completion_tokens": 5}
}]
}'

Analysis & agent

These spend model budget and accept either an API key or a session (owner/member). Viewers can read but not trigger analysis.

GET/api/v1/verdicts?traceId=key / session

The current verdict for a run (status, trace summary, confirmed issues). A read never triggers analysis: ingested runs are judged on arrival, so { verdict: null } means judging is still pending, or the run was examined by a monitoring sweep's audit, which records issues (GET /api/errors) rather than a verdict. Poll again or use POST /analyze. skipAnalysis=1 is accepted for compatibility (any other value is a 400) and changes nothing for an ingested run.

PATCH/api/v1/verdictskey / member

Record a trust signal on a verdict: { traceId, response: agreed | dismissed }.

POST/api/v1/traces/{id}/analyzekey / member

(Re-)run the judge for a trace. Async by default and durable: the run is queued as a job and 202 returns { requestId, jobId, status, statusUrl }. One active run per trace — a repeat POST while one is queued or running joins it (inFlight: true, with the original requestId) instead of spending twice. Pass ?wait=1 to block for the finished verdict. Idempotent, deduplicated within 30s.

GET/api/v1/traces/{id}/analyzekey / any member

The state of the latest analysis run for a trace: status none | queued | running | done | failed, the job (attempts, queuedAt, startedAt, updatedAt, lastError — e.g. a lease expiry when a runner was killed), and the current verdict. A done run whose verdict has status null and analysisFailedAt finished but could not judge the run. ?requestId= (from the POST response) pins the lookup to that run. This is how to tell a slow run from a dead one.

POST/api/v1/traces/{id}/investigatekey / member

Start a multi-step investigation thread for a trace. Returns { thread } with the thread id; read it back via GET /api/agent/threads/{threadId}. 202 { inFlight: true } while a previous investigation of the trace is still running.

GET/api/agent/threads/{threadId}key / session

Read an investigation thread back: status, messages, and findings. No spend, viewer-readable — the read-back counterpart of /investigate.

POST/api/agent/threads/{threadId}/messageskey / member

Ask a follow-up on an investigation thread: { message }. Resumes the loop and returns the updated { thread }; 202 { inFlight: true } while a previous follow-up is still running. Agent proposals can only be confirmed from the dashboard (session-only) — confirming spend-bearing actions requires a human.

GET/api/memory/citation-ratekey / session

How often investigations cite workspace memory: ?from=/?to= required (ISO, ≤92-day window). Returns { numerator, denominator, servedZero, rate } - finished threads shown at least one memory entry (denominator) vs. those that cited one (numerator); servedZero counts finished threads shown no memory, and rate is null when nothing was served. No spend, viewer-readable. The window is bounded by trace retention.

GETPOSTDELETE/api/v1/traces/{id}/chatkey / member

Streaming (SSE) investigator chat. GET rehydrates the thread; POST sends a message; DELETE clears it. GET is viewer-readable.

GET/api/v1/anomaly-findings?scanId=key / member

List persisted anomaly findings for a scan. Optional ?variant= filter; pass the literal value none to select workspace-wide findings. Regression investigations file under ?scanId=regression. Each finding carries clusterKey (the issue's identity across scans) and recursFindingId — the earlier finding under the same cluster key whose title it inherited, so one recurring issue keeps one name; group on clusterKey to count recurrences.

PATCH/api/v1/anomaly-findingskey / member

Dismiss or restore a finding: { findingId, dismissed: true | false }.

GETPATCH/api/v1/monitoringkey / member (GET), key / owner (PATCH)

Scheduled monitoring: sweeps, cadence, allowances, email digests, the Slack and Teams destinations (never the webhook URLs), and delivering - true when a connected channel receives each digest. PATCH is a partial update; chat delivery has no switch, connect a channel instead.

POST/api/v1/monitoring/teamskey / owner

Connect a Microsoft Teams channel from a Workflows webhook URL: { webhookUrl, label? }. Proven with a 'Kalmia connected' card before it is stored.

POST/api/v1/monitoring/slack-linkkey / owner

Start the Slack connect flow for a workspace owner: { ownerEmail }. Returns the 30-minute link that owner opens to pick the channel.

GET/api/annotations/detect-runs/{runId}key / member

Poll a batch behavior-detection run's progress and results.

POST/api/annotations/detectkey / member

Single-trace AI behavior detection; auto-annotates on a match. Spend is bounded by the agent rate-limit group and the daily spend budget.

POST/api/annotations/detect-batchkey / member

Async batch detection (≤500 candidates). Returns { runId, total }; poll detect-runs/{runId} for progress. Spend runs in background jobs, bounded by the rate-limit groups and the daily spend budget.

POST/api/v1/anomaly-scanskey / member

Trigger an anomaly scan: optional { variant?, from?, to? } scope/window (400 on a malformed window). Findings land on GET /api/v1/anomaly-findings.

POST/api/v1/regressionsmember

Investigate why one variant is worse than another: { variantA, variantB, windowStart?, windowEnd? }. Pins both populations (5 to 100 runs per side), files an investigating finding under scanId=regression, and runs a durable background investigation. Returns { findingId, existing }.

POST/api/traces/searchkey / member

LLM-ranked semantic search over the traces you supply: { query, traces, limit }, at most 100 traces. Returns scored results with reasoning and summaries. Costs credits by the number of traces supplied (402 when they cannot be covered); the session path requires the x-workspace-id header.

GET/api/v1/anomaly-scanskey / member

List past anomaly scans (newest first, up to 50): scope, window, status, finding counts. Trigger a scan via POST /api/v1/anomaly-scans.

GETPOST/api/v1/monitor-sweepskey / member (POST: owner)

GET: scheduled-sweep history with digests: window, status, each sweep's classified digest with delivery outcome, and coverage — tracesSeen, tracesTriaged, triageBudget, auditCoverage (0..1), triagePartial, and a hint when the audit budget capped triage below the window's volume (raise auditsPerDay in Settings → Monitoring; default 250/day, max 2000). Also returns monitoring: state (on | off), offSince, coveredThrough, nextDueAt, overdue, and gaps — the stretches no completed sweep covered, e.g. a pause. POST (owner): start one sweep by hand, at the price of one sweep. With no body it covers the traces since the last completed sweep (at most the last 7 days); with { from, to } it backfills exactly that window to close a gap (clamped to retention, allowances prorated by length). Both work whether or not scheduled sweeps are on. 402 when the owner's credits cannot cover a sweep, 409 while a sweep is in flight, 422 when there are no new traces to sweep.

GETPOST/api/prompt-audit/runskey / member

List prompt-audit runs, or start one: { variant, from, to, sampleSize } (sampleSize is silently clamped to 200). Returns { runId }. Viewers can read via GET.

GET/api/prompt-audit/runs/{runId}key / session

The full audit report: run status, checklists, per-trace verdicts with cited violations. POST /retry re-judges failed traces.

Issues

The grouped issue surface behind the Issues tab. Reads are no-spend, so viewers keep access. from and to are required ISO timestamps bounding a window of at most 366 days.

GET/api/errors?from=&to=key / session

One row per issue fingerprint in the window, worst severity and most runs first: { groups, facets }. Filters: variant, severity (low | medium | high), kind, source (comma list of signal, audit, judge), confirmed=1, dismissed=1. facets lists the variants, kinds, and sources present in the window regardless of the other filters.

GETPATCH/api/errors/{fingerprint}?from=&to=key / session (PATCH: member)

GET: { group, occurrences, fix, runs } - the aggregate (null when the window holds no occurrences), occurrences newest first (≤500), the cached fix if any, and each run's duration and cost. PATCH { dismissed: boolean } → { dismissedAt }; a dismissed group stays hidden until a newer occurrence lands.

POST/api/errors/{fingerprint}/fix?from=&to=[&force=1]key / member

Cached fix recommendation, or one structured judge call grounded in the window's occurrences and sample span content: { fix: { summary, likelyCause, fixes[{ title, detail, target: prompt | tool | code | infra }], model, generatedAt, sampleCount }, cached }. force=1 regenerates. 404 without occurrences, 502 when generation fails, 429 on a rate limit. Interactive judge quota.

Dashboard endpoints

These back the Kalmia dashboard and authenticate with your logged-in session and workspace membership. They are not reachable with an API key.

Traces

GET/api/tracessession

The dashboard trace list: one row per trace with a workspace-wide metrics rollup. A row is the trace's summary without its spans, plus spanCount and searchText (the distinct span names, tool names and error messages); the spans come with GET /api/traces/{id}. Keyset-paginated, newest first (?limit= default 100 max 500, ?cursor= → nextCursor); ?from=/?to= limit the list to a time window and ?variant= to one variant (repeat it for several, up to 50), alone or together, and both page by cursor the same way (truncated says the first page was not all of it); ?includeEval=true opts eval traces in.

GET/api/traces/{id}session

One trace in full for the active workspace (the x-workspace-id header): every span's complete content, tool arguments and results, run flags, and metrics - the heavy fields the list strips. The dashboard fetches it only for the trace you open. ?view=summary returns whole-run metrics and span previews instead, with lazySpans: true when the trace has more than 100 spans; ?spanOffset=N (a multiple of 100) then returns { spans, offset, total } with the full content of those 100 spans, 409 if the trace changed since the summary was read. 404 for an unknown trace or a workspace you are not in, never 403.

POST/api/traces/cited-spanssession

Resolve the spans a scan finding cites: { traceIds (≤200), spanIds (≤500) } → { spans: { [spanId]: { traceId, label } } }, read from those traces in the active workspace. A span none of the traces holds is left out. Read-only, no spend.

GET/api/traces/{id}/replay-linksession

{ replayUrl } for the trace's session recording in PostHog, or null when there is nothing to link (no completed replay row, replay off, or an unknown trace - the null shape is the common case, not an error). Requires the x-workspace-id header.

GET/api/traces/{id}/replay-recordingsession

{ events } - the stored rrweb recording of a Kalmia watchable capture, decoded server-side so the in-app player receives plain JSON. 404 when the trace has no recording. Requires the x-workspace-id header.

Evals

The eval editor and scoreboard. All eval routes are deliberately session-only: an API-key runner submits traces and reads grades over MCP, but can never trigger judge spend or influence the scores it is graded by. Reads admit viewers; mutations require owner or member.

GETPOST/api/evals/casessession / member

List cases with criteria counts, or create one manually (duplicate externalId → 409). Cap: 20 effective criteria per case.

GETPATCHDELETE/api/evals/cases/{caseId}session / member

Full case editor model in one fetch; rename (externalId is immutable); delete — 409 if graded results reference the case.

POSTDELETE/api/evals/cases/{caseId}/behaviorsmember

Attach or detach a workspace behavior, with expectation occurs | does_not_occur. Idempotent; attach 422s past the cap.

POSTPATCHDELETE/api/evals/cases/{caseId}/criteriamember

Add, edit, and delete case criteria (≤2,000 chars each). Edits never rewrite past results.

POST/api/evals/cases/{caseId}/criteria/reordermember

Reorder a case's criteria: { orderedIds }, which must be a permutation of the case's current criterion ids. A stale editor (a concurrent add or remove) gets 400 rather than silently corrupting positions.

GET/api/evals/runssession

Recent run sets, newest first, with derived outcome counts. ?limit= up to 100 (default 20), ?offset=.

GET/api/evals/runs/{runId}session

The full run report with inline evidence — readable even after the underlying traces expire.

GET/api/evals/compare?base=&candidate=session

Fixed / regressed / both-pass / both-fail comparison of two run ids, with a judge-model-changed badge.

POST/api/evals/results/{resultId}/diagnosemember

On-demand failure diagnosis for a failed result. 409 if the trace expired or the result is not failed.

POST/api/evals/results/{resultId}/rescoremember

Re-enqueue judging for unscored criteria (202). Reviewer-touched scores are never cleared.

POST/api/evals/scores/{scoreId}/reviewmember

Confirm or override one criterion score: { action: 'confirm' } or { action: 'override', pass }. Latest action wins.

Search

POST/api/semantic-searchmember

Judge traces against a natural-language behavior query. { query, traceIds, limit ≤200 }: the server summarizes each stored trace (up to limit of them, ≤1,000 ids per request). { traceSummaries } instead supplies pre-built summaries. Spends judge budget.

Workspace & collaboration

GET/api/workspaces/{id}/membersowner

List members.

PATCHDELETE/api/workspaces/{id}/members/{userId}owner

Change a member's role, or remove them. The last owner cannot be demoted or removed.

GETPOST/api/workspaces/{id}/invitesowner

List invites, or send one by email (member/viewer only).

DELETE/api/workspaces/{id}/invites/{inviteId}owner

Revoke a pending invite.

POST/api/invites/acceptsession

Accept pending invites matching your verified email.

GETPATCH/api/workspaces/{id}/settingsmember / owner

Read the workspace settings in one fetch: judge model (with the picker's options and whether it is the default), judge engine, session-replay config, scheduled-sweep config, your role, and the current rate-limit status. PATCH (owner) sends exactly one of judgeModel, judgeEngine, rateLimitOverride, replayConfig, or sweepConfig; each setting updates independently, so changing one never resets another.

GETPOST/api/workspaces/{id}/githubmember / owner

Read the GitHub connection state (?include=repos for the picker), or select the connected repository (owner). Connect via Settings → GitHub.

GET/api/github/connect?workspace=owner (browser)

Starts the GitHub App install: mints a short-lived state token bound to the workspace and your session, then redirects your browser to GitHub's install page. A navigation, not a fetch; failures redirect back to settings with an error code.

GET/api/github/setupbrowser callback

GitHub's post-install callback. Nothing is stored until the state verifies, the live session matches the user who started the flow, that user is still an owner, and the completing user is proven to have access to the installation.

GETDELETE/api/workspaces/{id}/slackmember / owner

GET: the Slack destination as any member may see it - connected, channel, team, and whether the webhook is broken; the webhook URL itself never crosses this boundary. DELETE (owner): disconnect; digests stop posting to Slack from the next sweep.

POST/api/workspaces/{id}/slack/connectowner

Start the Slack connect flow: returns { url }, the Slack authorize page bound to this workspace and your session; the settings page navigates there and Slack's own consent screen picks the channel. 422 when Slack is not configured on the server.

GETPUTDELETE/api/workspaces/{id}/teamsmember / owner

The Microsoft Teams destination. GET: connected, label, and whether the webhook is broken; the webhook URL never crosses this boundary. PUT (owner): { webhookUrl, label? } - a Teams Workflows ('Send webhook alerts to a channel') webhook URL the owner pasted; the host is checked, a 'Kalmia connected' card is posted to prove it works, and only then is it stored encrypted (400 for a non-Workflows URL, 422 when Teams rejects or cannot be reached). DELETE (owner): disconnect; digests stop posting to Teams from the next sweep.

GETPOST/api/workspaces/{id}/contextsession / member

List domain-context documents, or create one from text or markdown: { name (≤200 chars), text (≤200,000 chars), sourceType }. The document is chunked and embedded in the background and reports processing → ready | failed.

GETPUTPATCHDELETE/api/workspaces/{id}/context/{docId}session / member

GET the document's editable name and raw text; PUT edits name and/or text (a text change re-embeds; a throttled re-embed returns 429); PATCH { active: boolean } switches it on or off without deleting; DELETE removes it. Viewers can read; writes need owner or member.

POST/api/agent/proposals/{proposalId}member (session only)

Resolve a change the trace investigator proposed: { action: 'confirm' | 'decline' }. Deliberately session-only with the x-workspace-id header, so a machine can never satisfy the human-confirmation step. Exactly one of concurrent confirms applies; a lost race gets 409 with the actual resolution, and a foreign or unknown id gets 404.

POST/api/traces/{id}/flags/{flagId}/fix-recmember

Generate (or retry) a repo-grounded fix recommendation for a confirmed run flag. 422 without a GitHub integration, 409 without a connected repo.

GET/api/workspaces/{id}/auditowner

Query and export the workspace audit log.

API key lifecycle

GETPOST/api/v1/api-keysowner

Read key metadata (never the key), or create a key: { name?, scope? } where scope is 'full' (default) or 'ingest' (write-only). Raw key shown once; 409 if one exists.

POST/api/v1/api-keys/rotateowner

Atomically revoke all active keys and mint a replacement. Preserves the current scope unless a new one is passed.

Rate limits

Rate limiting is scoped per (operation group, workspace) - see Rate limits for the groups and defaults. Billable model calls (judging, detection, investigation, chat) draw from their group budgets and from the workspace owner's spend budget (actual token cost across every workspace they own, limited per day and per month); ingestion (SDK, OTel, feedback) passes a deliberately generous per-request admission gate (default 300 requests/min per workspace) that normal traffic never hits and is never blocked by the spend budget. Dashboard reads and the key-authenticated read/annotation endpoints (trace list and detail, analytics, annotations) are not currently rate limited. When a limit is exceeded, the response is:

HTTP/1.1 429 Too Many Requests
RateLimit-Policy: burst;q=120;w=60, sustained;q=5000;w=86400
RateLimit: burst;r=0;t=42
Retry-After: 42
{ "error": "Rate limit exceeded", "requestId": "…", "retryAfter": 42 }

Under-limit responses carry no rate-limit headers. If the limiter backend is briefly unavailable, calls fail open (allowed through) rather than blocking your traffic.

MCP server

Kalmia exposes a Model Context Protocol server so your own coding agent can pull eval results, query the trace corpus, read verdicts and anomaly findings, and — with your approval — run the judge, investigate, and tag runs, all inside your repo. It's authenticated with your workspace API key and returns results as labeled data, never as instructions.

claude mcp add --transport http kalmia https://www.kalmia.dev/api/mcp \
--header "Authorization: Bearer kal_live_your_api_key"

Tools

ToolReturns
list_recent_runsRecent eval run sets with per-status case counts
get_run_resultsCursor-paginated case results for a run, with status counts
get_case_diagnosisThe root-cause diagnosis for a failed case
compare_runsFixed / regressed comparison of two run ids
query_tracesTraces filtered by metric ranges, attributes (including variant), status, and time window
aggregate_tracesPer-group counts and rates over the filtered corpus (by model, agent, variant, status, or tool)
compare_metricsSide-by-side metrics per variant, primary model, or project (duration, tokens, error rate, cost); can compare models within one variant
get_traceOne run's slim summary with metrics, annotations, and customer feedback
search_trace_contentSpan ids + snippets where a keyword appears inside one trace
get_span_detailOne span's full content: complete input/output, tool args and result (paged)
get_verdictA trace's current verdict: status, trace summary, confirmed issues, suggested fix
list_anomaly_findingsPersisted detector findings: severity, type, description, trace ids (optionally only those citing one variant)
list_issuesThe Issues tab over a window, filterable by variant, kind, and severity; with compareVariant, which issues are new, shared, or gone between two variants
get_issueOne issue: run counts, variants, its newest runs (optionally one variant's), and the cached fix
compare_variants2-5 variants side by side, baseline first: failure rate, failed-verdict rate, behavior prevalence, anomaly findings, new / gone issues, and average cost, tokens, and duration, with the change against the baseline
analyze_traceACTION: run the judge on one trace and return the verdict
investigate_traceACTION: start or continue a multi-step investigation of one trace
annotate_traceACTION: apply a behavior label to one trace (idempotent, reversible)
get_monitoring_settingsScheduled sweeps, email digests, the Slack and Teams destinations, and whether digests are posting to a channel
run_sweepACTION: run one sweep now over the traces since the last one; costs the credits of one sweep
configure_monitoringACTION: turn scheduled sweeps and digest email on or off (partial update); channel delivery follows what is connected
connect_slackACTION: a 30-minute link a workspace owner opens to approve the Slack channel for digests
connect_teamsACTION: store a pasted Microsoft Teams Workflows webhook for digests, after a test card proves it works

The action tools are the only ones without readOnlyHint, so MCP hosts (Claude Code and peers) ask for your approval before running them. The three trace actions each act on exactly one trace — there is deliberately no bulk tool — and their spend rides the same rate-limit groups and daily spend budget as every other trigger; a budget refusal returns a structured rate_limited result with retryAfterSeconds. The three monitoring actions change workspace settings only: they never touch trace data, and connect_slack cannot finish on its own — Slack requires a signed-in workspace owner to click Allow, and the callback refuses any other session. What an agent can never do here: score, override, or change eval outcomes, agree/dismiss a verdict, or alter customer feedback — the actor being graded never holds the pen. See Running evals → close the loop for the full workflow.