Study pack set
Every AI study tool is tested on the same material: 60 questions whose answers I worked out first, a 12-page study pack that holds the 60 facts, a ten-fact fact sheet and an invented lecture with ten key points.
- When it runs
- At the start of every study tool review
- Used in
- 8 reviews
- Runs on
- A script in this site's repository, run on my PC
What the study pack set is
The facts are real and checkable, 20 each on maths and science, humanities and general knowledge, so every answer can be marked right, wrong or invented. The lecture and its lighthouse are made up, so no tool has seen them.
A tool that answers from the documents you give it, as NotebookLM does, is asked the questions with the pack loaded as its only source. A tool that stops early, because its free allowance runs out, is graded on the questions it answered and the review says how many.
How it works
- A script writes the study pack as a PDF, a Word file and a text file, with the answer key, the fact sheet, the lecture and its ten key points.
- The same files go to every tool and nothing is changed between tools.
- Each answer is marked against the key by hand, and a wrong or invented answer is listed.
What it records
- Per tool
- The questions asked, each answer, the time it took, and the count right, wrong and invented.
What it cannot catch
- The pack answers every question, so it tests answering from a document, not what a tool knows without one.
- A free allowance may stop the run before the 60 questions are done.
Where it was used
Every published review that ran this checker, with the figure it produced there, read from the review’s own data. Each figure opens the test results behind it.
| Review | Figure | Tested |
|---|---|---|
| 60 of 60 Questions asked | 7 Nov 2026 | |
| 59 of 60 Questions right | 6 Nov 2026 | |
| 60 of 60 Questions answered correctly | 6 Nov 2026 | |
| 20 of 60 Questions answered and seen | 6 Nov 2026 | |
| 49 of 49 answers right | Invalid Date | |
| 11 of 60 Questions asked | 6 Nov 2026 | |
| 5 of 60 Questions answered and seen | 6 Nov 2026 | |
| 10 of 60 Questions answered and seen | 6 Nov 2026 |
Review
- Figure
- 60 of 60 Questions asked
- Tested
- 7 Nov 2026
- Figure
- 59 of 60 Questions right
- Tested
- 6 Nov 2026
- Tested
- 6 Nov 2026
- Tested
- 6 Nov 2026
- Figure
- 49 of 49 answers right
- Tested
- Invalid Date
- Figure
- 11 of 60 Questions asked
- Tested
- 6 Nov 2026
- Tested
- 6 Nov 2026
- Tested
- 6 Nov 2026
The other checkers for ai study tool
- Study question set
- Study source quiz check
- Credit ledger
- Question runner
- Upload checker
- Study source pack
- Study upload checker
- Study phone check
- AI study tools methodology
- All checkers
Written by Miles Foster, Owner and tester, RankDude. How RankDude tests and grades is on the methodology page.