RankDude

Assistant test pack

To compare assistants fairly, each one is asked the same things about the same invented person, and each answer is checked against an answer key I wrote before the test.

Checker · AI personal assistant
When it runs
At the start of every review
Used in
1 review
Runs on
My PC

What the assistant test pack is

The person, the facts, the requests and the answer keys are one file in the methodology folder. The two supplied files are made by a script, so they are the same every time: a month of spending with a known total, and a building notice with ten planted facts.

How it works

  1. The facts are told once, in one chat; every later request is typed in a new chat without repeating them.
  2. Each answer is checked in the real app where there is one, such as the Automations list, and otherwise against the answer key.
  3. A request counts as done only when it needed no correction from me.

What it records

Per request
The request, the answer, what was checked and whether it was done.

What it cannot catch

  • The names and numbers are invented, so an assistant cannot know them from anywhere else.
  • A free plan with a daily message cap runs fewer than the full thirty requests; the review says how many.

Where it was used

Every published review that ran this checker, with the figure it produced there, read from the review’s own data. Each figure opens the test results behind it.

Review

The other checkers for ai personal assistant

Written by Miles Foster, Owner and tester, RankDude. How RankDude tests and grades is on the methodology page.