RankDude

Study pack set

Every AI study tool is tested on the same material: 60 questions whose answers I worked out first, a 12-page study pack that holds the 60 facts, a ten-fact fact sheet and an invented lecture with ten key points.

Checker · AI study tool
When it runs
At the start of every study tool review
Used in
8 reviews
Runs on
A script in this site's repository, run on my PC

What the study pack set is

The facts are real and checkable, 20 each on maths and science, humanities and general knowledge, so every answer can be marked right, wrong or invented. The lecture and its lighthouse are made up, so no tool has seen them.

A tool that answers from the documents you give it, as NotebookLM does, is asked the questions with the pack loaded as its only source. A tool that stops early, because its free allowance runs out, is graded on the questions it answered and the review says how many.

How it works

  1. A script writes the study pack as a PDF, a Word file and a text file, with the answer key, the fact sheet, the lecture and its ten key points.
  2. The same files go to every tool and nothing is changed between tools.
  3. Each answer is marked against the key by hand, and a wrong or invented answer is listed.

What it records

Per tool
The questions asked, each answer, the time it took, and the count right, wrong and invented.

What it cannot catch

  • The pack answers every question, so it tests answering from a document, not what a tool knows without one.
  • A free allowance may stop the run before the 60 questions are done.

Where it was used

Every published review that ran this checker, with the figure it produced there, read from the review’s own data. Each figure opens the test results behind it.

The other checkers for ai study tool

Written by Miles Foster, Owner and tester, RankDude. How RankDude tests and grades is on the methodology page.