Checker · AI agent

Ledger verifier

Before an AI agent review goes live, the verifier recomputes every figure in it from the raw ledger and the credit reads, compares each with what the review says, and fails if one does not match.

When it runs: Before a review is published, and again after every edit
Used in: no published review yet
Runs on: The ledger and the review's data, on my PC

A review is edited many times between the test and publication, and a figure copied once can drift. The verifier treats the review as a set of claims: each task's minutes and credits, the task counts, the false dones, the median time, the credits used on the test day, the file points, the grades and the charts. It recomputes every claim from the source files and prints the two values side by side.

How it works

It runs the output checker over every task and reads the task ledger and the credit reads.
From those it recomputes: the tasks finished by the checker and by the ledger, the false dones, the median minutes per task, the credits used between the first and the last read of the test day, the points scored over the three file tasks, the figures the agent guessed, and the invented contact details.
It reads the review's own data files and compares every claim: each task row's minutes and credits within a small tolerance, each task's result, the counts and the grades they produce, the average, the test-day credits within a small margin, and one consistent price per credit wherever money is quoted.
Every chart block in the review is handed to the chart check, which confirms the chart's figures against the same sources.
Each claim ends as a match, a mismatch, a note, or no data. One mismatch fails the run and the review is not published until it is fixed.

What it records

Per claim: What the review says, what was recomputed, and the status: match, mismatch, note or no data.
Per run: The number of claims, mismatches and notes, and every recomputed figure.

What it cannot catch

It can only recompute what the ledger and the checker hold. Grades that rest on the live product, such as memory, support, autonomy and safety checkpoints, are checked by hand.
The price of a credit comes from the checkout, so a wrong price there would pass as long as it is used consistently.

Where it was used

Built and checked; the first review to use it will appear here when it is published.

The other checkers for ai agent

Written by Miles Foster, Owner and tester, RankDude. How RankDude tests and grades is on the methodology page.