# Understanding variant metrics

A glossary of the metrics Kalmia computes for every run, and how to read them when comparing variants.

## Core metrics

Computed for every trace and aggregated per variant.

**Duration**: Wall-clock time from the first span to the last — end-to-end latency of the run.

**Total tokens**: Sum of prompt and completion tokens across every LLM call in the trace.

**Input / output tokens**: Prompt vs. completion tokens. High input often means large context or long history; high output often means verbose responses.

**Turns**: Number of LLM conversation turns — each assistant response is one turn.

**Cost**: Estimated USD from token counts at list prices (see [Reading cost](https://www.kalmia.dev/docs/analytics.md#cost)).

## Agent metrics

Extra signals for tool-using and coding agents.

- **Tool calls**: Total tool invocations, broken down by name in the trace
- **Tool errors**: Tool calls that returned an error
- **Retries**: Repeated attempts at the same operation — a sign the agent is stuck
- **Backtracks**: Times the agent undid a previous action
- **Files read / written**: Unique files touched during the run
- **Subagent spawns**: Times work was delegated to a sub-agent

## Failure tags

Traces are automatically tagged with failure patterns during preprocessing, so you can surface problems without reading every run:

- **permission_denied**: Permission or access errors
- **file_not_found**: Referenced files that don't exist
- **high_token_usage**: Token count above the normal range
- **excessive_retries**: Retried the same operation too many times

## Reading deltas

In the comparison table, one column is the baseline and every other column shows its change against it; choose the baseline from the dropdown in the first column.
The best and worst value in each row are highlighted.
Read the direction with the metric: lower duration, tokens, errors, and retries are better; higher success rate is better.
The same table can group runs by model or project instead of variant; see [Compare by model](https://www.kalmia.dev/docs/guides/comparing-variants.md#by-model).

> **Watch the combinations:** High retries plus high token usage is a strong signal a variant needs work. Prefer the median over the average for latency and tokens — outliers skew averages.
