Tasks, outcomes, contracts and scorecards

Metrx reads an agent the way you would read anyone doing a job: what it was asked to do, what came of it, what it cost, and how that compares with what you expected. Four objects carry that, in this order:

task → outcome → contract → scorecard

Task

A task is one unit of the agent’s work: a ticket handled, a document extracted, a reply drafted. It is identified by a task_id, a UUID, stamped on every LLM call made for it.

• TypeScript SDK: task(client, fn) runs fn in a task context. The task_id is a v4 UUID unless you pass your own with { taskId }. A UUID is kept as is. Mapping any other string to the same UUIDv5 as the Python SDK needs @metrxbot/sdk 0.7.0; on 0.6.0 a non-UUID taskId correlates nothing, so pass a UUID.
• Python SDK: m.task(...) opens the same context. Calls made inside the block carry its task_id.
• Gateway: send the X-Task-Id header yourself.
• OpenTelemetry: a span attribute metrxbot.task_id names the task; without one, the trace id is the task.

A task with no outcome is still valid: its calls are correlated, which is enough for cost per task.

Outcome

An outcome is the result of a task, posted for its task_id:

POST https://app.metrxbot.com/api/v1/tasks/{task_id}/outcome

The SDKs post it for you when the task block ends after setOutcome(...) (TypeScript) or set_outcome(...) (Python). The body is success (required), plus optional value_microcents, category, metadata and notes. If the block throws, the SDK posts a success: false outcome unless you turn that off.

Rules the endpoint enforces:

• The task_id must be a UUID; anything else is refused with 400.
• The task_id must appear on at least one of your organization’s LLM calls. That is how Metrx knows the task is yours, and it is where the outcome gets its agent. An outcome that arrives before its calls is held for up to 24 hours and returned as 202 with status: "pending_correlation"; it is written when the calls land. This is why the TypeScript snippet flushes the exporter before posting the outcome.
• Posting again for the same task_id updates the outcome rather than adding a second one.

An SDK-posted outcome lands in the human_verified confidence tier, which means “confirmed by a person in Metrx or asserted by your own SDK”. Metrx cannot tell whether a person reviewed an SDK-asserted outcome, and the scorecard labels the tier “Confirmed” for that reason.

Contract

A job contract states what the agent is for and what counts as doing it well. It is set per agent with:

POST https://app.metrxbot.com/api/v1/agents/{agent_key}/contract

or from the agent’s Contract tab in the dashboard (org admins and owners). The POST needs an admin session or an API key with the admin scope; an ordinary read/write key gets 403 insufficient_scope. GET on the same path returns the current version and every earlier one.

The six job fields are required on every version:

op is one of >=, >, <=, <, ==. A window is a rolling window such as 24h, 7d or 4w. An optional human_baseline (cost_per_unit_microdollars, optional minutes_per_unit, and a required basis) records what a person’s unit of the same work costs.

Versions are append-only. Each POST publishes a new version numbered one above the highest ever written, and supersedes the current one; nothing is edited in place. Engine terms (the policy that says what Metrx may change for this agent) carry over from the current version unless the body supplies them.

The first version must say where its engine terms come from, with policy_defaults:

• observe_only: terms that grant no authority. Every action class is shadow, so Metrx collects evidence and changes nothing. This mode also needs task_verb (one of extraction, generation, retrieval, analysis, classification, review, calculation, orchestration) and domain (one of finance, legal, compliance, healthcare, marketing, sales, support, engineering, data, general).
• inherit_workload_policy: copies the organization’s own active policy for the workload, as a snapshot. It is refused when no such policy exists.

With neither, every engine term must be in the body.

Scorecard

The scorecard reads the agent’s measured figures against its contract: one check for the KPI target, one per acceptance criterion, and one for the budget. Each check is pass, fail or insufficient.

A contract metric is free text, but only these names are computed. Any other name is shown as not measured, never guessed at:

No rate until 20 outcomes. A rate or per-outcome figure over fewer than 20 outcomes in its window is insufficient. That minimum is Metrx’s rule, not a contract term.

A check is also insufficient, not a zero, when:

• no calls and no outcomes reached Metrx in the window (an idle agent and an uninstrumented one look the same);
• a spend term has no metered calls (an absent call is not $0), or an outcome term has no outcomes;
• the window is longer than 400 days, or longer than your plan’s history.

The budget check reads spend over a rolling window: daily is 24h, weekly is 7d, monthly is 30d. The human baseline, when set, is shown beside the measured spend per outcome and is not graded, because an outcome is not guaranteed to be the same unit of work as a person’s.

The scorecard as a whole is fail if any check fails, insufficient if none fails but at least one is insufficient, and pass otherwise.

Each unit above, as plain arithmetic with a worked example: How Metrx counts.