> ## Documentation Index
> Fetch the complete documentation index at: https://docs.algovoi.co.uk/llms.txt
> Use this file to discover all available pages before exploring further.

# ATB vNext: verifiable agent evaluation

> A verifiable, offline, multi-mode evaluation harness for payment agents. Six test modes, seven scenario packs across the OWASP LLM Top 10, five scores, and a signed, reproducible corpus. Every result is a content-addressed, Falcon-signed artefact you can recompute yourself.

The Agent Trust Bench vNext is an open harness for testing agents that authorise and settle
payments. It goes beyond adversarial prompts into structured **test modes**, selectable
**scenario packs**, and multi-dimensional **scoring**, and every result is a content-addressed
Keystone reference (`sha256:` + SHA-256 of RFC 8785 JCS) so it recomputes byte for byte, offline.

## Install

```bash theme={null}
pip install --extra-index-url https://pip.algovoi.co.uk/simple/ algovoi-atb-vnext
algovoi-atb-vnext verify      # verify the signed conformance corpus, offline
algovoi-atb-vnext coverage    # OWASP / mode / severity coverage
```

## Test modes

| Mode         | What it checks                                                             |
| ------------ | -------------------------------------------------------------------------- |
| `unit`       | one decision in isolation (tool choice, parameter, refusal)                |
| `workflow`   | a full flow: quote, consent, authorisation, settlement, receipt            |
| `persona`    | multi-turn dialogue under ambiguity, pressure, social engineering          |
| `redteam`    | adversarial prompts, policy conflicts, manipulated tool output             |
| `runtime`    | scores in-flight and halts on a severity breach, not only at the end       |
| `regression` | replays fixed goldens so drift is attributable to the exact ref that moved |

## Scenario packs

Seven selectable packs, not one monolith: `tool-misuse`, `aml-edgecase`, `persona`, `long-context`,
`multi-agent`, `data-protection`, `runtime`. Together they exercise all seven of the OWASP LLM Top-10 categories the
bench maps (LLM01, LLM02, LLM05, LLM06, LLM07, LLM08, LLM09).

## Scores

Five per run, each content-addressed: **outcome** (ended safely and correctly), **trace** (each
step justified and policy-aligned), **trust** (per-turn grounding and honesty), **cost/latency**,
and **severity-weighted** (an unsafe success is penalised far more than a safe refusal, so
"passed but dangerously" fails harder than a cautious block).

## Verify it yourself

Every scored run is an `atb_run_v2` object whose `run_ref` is the SHA-256 over its canonical JSON,
Falcon-1024 signed by the bench. Recompute the ref from the object's own fields to confirm it, and
run `algovoi-atb-vnext verify` to check the signed corpus (positives recompute, tamper negatives
diverge) against the AlgoVoi Ed25519 corpus key. No trust in the bench is required; the maths is
open and the two standards (RFC 8785, SHA-256) are all you need.

## Selectable, identity-bound suites

Pick a subset of tests and run them as one suite bound to your agent's identity, so the result
composes with the reputation the agent already holds:

```bash theme={null}
# preview a selection, then run it
curl "https://agent-trust-bench.algovoi.co.uk/agent-trust-bench/vnext/select?packs=tool-misuse,aml-edgecase"
```

The suite result shares the same subject as the agent's ATB Pass Certificate and is itself a
signed, recomputable reference.