Agent Output Verification

How do I verify my AI agent's outputs?

Your agent just told a customer that a company's CEO is someone who left the job months ago. It said it confidently. It even produced a citation. The citation was real; the fact was stale. Nobody noticed until the customer did.

This is the shape of the problem: agents fail confidently, and self-checking doesn’t work — a model grading its own answer inherits its own blind spots. During the calibration of our verification engine, the engine flagged a label we had written as wrong. We’d said Berkshire Hathaway’s CEO was Warren Buffett. The engine went to primary sources and refuted its own supervisor: Greg Abel had the job. The labeler’s world-model was frozen; the verifier’s wasn’t. That incident is the whole thesis in one line: verification has to be adversarial and external, or it’s decoration.

What verification actually requires

Three properties, each testable:

1. Independent evidence, both directions. A verifier that only searches for support will find it — the web agrees with almost anything. Ours runs a gather-for and a gather-against pass, then a dedicated refuter whose only job is to kill the claim. A claim survives, gets refuted, or — critically — comes back UNVERIFIABLE when the evidence isn’t there. An honest “I can’t verify this” outranks a confident guess: on our measured pilot, every fabricated or private claim we threw at the engine was refused, not graded.

2. A measured calibration curve. “Confidence 0.9” means nothing unless 0.9-confidence verdicts are right ~90% of the time, measured on labeled data. Our production curve — measured on the retrieval stack that actually serves requests — is top-bucket realized accuracy 0.914 (n=116) at 27% abstention, with decidable-commit precision 0.976 across a 204-claim labeled set.

And here is the part worth stealing: we measured 0.909 (n=121) on our home research substrate, and we do not publish that number, because a curve measured on one retrieval stack does not transfer to another. Swapping the gather implementation is enough to invalidate it — the keyless stack lands at 0.848 (n=79). Any verification vendor quoting a single accuracy figure without naming the stack it was measured on is quoting the best of several. Ask which one is serving your requests. When the engine hadn’t earned a curve yet, it was hard-capped and the API served a labeled stub — no paid verdict without a measured curve. That gate is the rule we’d suggest for any verification you build or buy: ask the operator “what’s your realized accuracy at the confidence you’re quoting, and on what n?” If there’s no number, it’s vibes.

3. Receipts you can’t quietly rewrite. Every verdict we sell is an Ed25519-signed receipt in a hash-chained, append-only ledger — anyone can check the chain at /ledger/verify, and a tampered line breaks it publicly. If your agent’s decisions matter enough to verify, they matter enough to prove later.

The build-vs-buy honest version

You can build the first property yourself in a weekend: prompt one model instance to refute another’s answer before you act on it, and treat “can’t refute AND primary source found” as your bar. That alone catches the worst failures. What’s expensive to build is the second and third properties — labeled calibration sets, measured curves, signed receipt chains.

That’s the part we sell: one HTTP call, $0.05, paid by your agent itself over x402 — verify.clearsigned.com. It returns SUPPORTED / REFUTED / UNVERIFIABLE with evidence, confidence backed by the published curve, and a signed receipt. Or start free: check whether your own site is even legible to agents at clearsigned.com/scan.

Clearsigned is verification infrastructure for the agent economy. The judge doesn’t litigate: we never compete to supply the work we verify, and customer payloads are never training data.

How do I verify my AI agent’s outputs?