RankDude

How I test AI study tools

Ten tests, the same for every study tool, each graded from 1 to 5 on a fixed scale. The overall grade is their average plus my verdict, which can move it by up to one point. Every test is run by hand on an account I use, and the questions, the source documents, the generated cards and quizzes, the counts, screenshots and timings behind each grade go in the full test results published with the review.

Tested and written by Miles Foster, Owner and tester, RankDude.

Each study tool gets the same material, built before any tool is opened: a fixed set of 60 questions with answers worked out and checked in advance, 20 each from maths and science, humanities (history, literature, civics) and general knowledge, every one with a single known-correct answer so a reply can be marked right, wrong or invented rather than admired; one 3,000-word lecture transcript and one 12-page PDF chapter, each with a list of the key points written out before the tool sees them, used for the summarising and flashcard tests; and one short source document with ten checkable facts in it (names, dates, figures) so a generated quiz or set of flashcards can be marked for whether it kept the material straight. The same questions, documents and source go to every tool in the category, so two tools are judged on the same work.

Each study tool is run on the best plan the account offers, on a free or trial account where one exists, at the default settings, with each question asked once and the first answer taken, and each document uploaded once. Where a tool teaches through a back-and-forth rather than a single reply, the first complete answer or the end of the first guided exchange is what counts. Nothing is graded from a vendor's accuracy claim, an app-store rating or a marketing page. Each review states how the account was obtained: used on a free tier, bought at the public price, or provided by the vendor. A vendor-provided account never changes a grade, and a vendor never sees a review before it is published.

How I rate

Every tool gets one score out of 5.

First, I run the same 10 tests on every tool and give each one a grade from 1 to 5. The average of the graded tests is the starting point; a test that cannot be graded is left out of it, and the review says how many tests the score is based on.

Then I add my verdict: what the tests cannot measure, such as how the tool holds up after the test weeks, how billing and cancelling work in practice, and how it compares with the others in the category. My verdict can move the score up or down by one point at most.

Every review shows how much my verdict changed the score, and why, in its scores table.

What a grade means

5Excellent
4Good
3Average
2Below average
1Poor

The final score is the average of the graded tests (ten when every test can be graded), to one decimal place, plus my verdict, which can move it by up to one point. If a test couldn't be finished properly it gets no grade, and the review says how many tests the score is based on. A tool with no file upload, no quiz generator or no mobile app is not penalised on a capability it does not claim; that test is scored on what it can attempt, with the count stated, or left out and noted. A tool that answers fast but invents facts, or that generates tidy flashcards full of wrong answers, is marked down hard, because a study aid that teaches the wrong thing is worse than none. Some tests on a free plan use fewer questions or a shorter document than usual because of daily or upload limits; I run them again on a paid plan when I can.

The 10 tests

1.Answer & Explanation Accuracy

The 60 questions and their answers are written and checked before any tool is opened, 20 each across maths and science, humanities and general knowledge. Each question is asked once and the first complete answer is taken and marked right, wrong, or invented, where invented means a confident answer that is simply made up (a wrong formula stated as fact, a fabricated date, a cited quotation that does not exist). The count correct out of 60 is the grade, and the invented answers are reported separately and counted, because a wrong answer a student can catch is a smaller harm than a confident fabrication they will copy. The subjects the tool is strongest and weakest on are noted; the per-subject split is graded on its own in test 6.

  • 5: 54 to 60 of 60 correct and nothing invented
  • 4: 48 to 53 correct
  • 3: 39 to 47 correct
  • 2: 27 to 38 correct
  • 1: fewer than 27 correct, or confident fabrications throughout

2.Flashcards & Quizzes From a Source

The fixed source document, which carries ten checkable facts written in advance, is given to the tool and it is asked to make a set of flashcards and a quiz from it. The first 20 generated items (cards and questions together) are read and each is marked usable only when it is factually correct, has a single answerable answer, and is actually drawn from the source rather than invented or off-topic. A card with a wrong answer, a quiz question with no correct option or two correct options, or an item about something not in the source is a failure, and each is listed. Whether the ten planted facts were picked up, and whether the answer key the tool supplies is itself correct, are reported, because a quiz that marks the right answer wrong teaches the mistake.

  • 5: 18 to 20 of 20 accurate, answerable and on-source
  • 4: 15 to 17
  • 3: 11 to 14
  • 2: 6 to 10
  • 1: 5 or fewer, or a generator that invents answers or cannot use the source

3.Summarising a Lecture or PDF

Each of the two documents has ten key points listed before the tool sees it. The lecture transcript and the PDF chapter are each uploaded or pasted once and summarised at the default length, and the summary is read against the list: a point counts as kept when it is present and correct, and any statement the summary adds that the source does not support is counted as an error introduced. The grade is set by the weaker of the two documents (points kept, minus errors). The reading level and length of the summary, whether it keeps headings and structure, and whether it quietly changes a figure or a name from the source are reported, since a summary that is wrong is worse than a long one.

  • 5: 9 or 10 of 10 points kept on each document, no error introduced
  • 4: 7 or 8 kept, at most one error
  • 3: 5 or 6 kept
  • 2: 3 or 4 kept, or errors introduced on both
  • 1: 2 or fewer kept, or the summary misstates the source

4.Step-by-Step Explanations

Twenty problems that have a method, not just a fact (a maths problem, a science calculation, an essay-structure or source-analysis task), are each asked once with a request to explain the steps. An explanation counts only when the working is correct, is laid out in steps a student could follow and reproduce, and leads to the right answer; a bare answer with no working, a jump that skips the hard step, or correct-looking steps that reach a wrong answer is a failure. The count of genuine step-by-step explanations out of 20 is the grade. Whether the tool stays in a teaching mode (asking the student to try, giving hints) or simply solves the problem outright is reported, because a tool that only gives answers is a homework-doer, not a study aid.

  • 5: 17 to 20 of 20 teach correct, followable steps
  • 4: 14 to 16
  • 3: 10 to 13
  • 2: 6 to 9
  • 1: 5 or fewer, or it only returns bare answers

5.Uploaded Document & Notes Handling

Each point is checked by doing it on the test date: paste notes as text; upload a PDF and have the tool use it; upload a DOCX; upload a slide deck (PPT or Google Slides); upload or photograph an image of handwritten or printed notes; bring in audio or a recorded lecture; bring in a YouTube or video link; handle a long document (30 pages or more) on the plan tested without truncating it; keep several documents in one workspace or notebook the tool can draw across; and export or download what it generates (cards, notes, quiz). A point counts only when it works end to end on the account used, not when it is merely listed on a plan I am not on, and each is screenshotted.

  • 5: all ten points
  • 4: 8 or 9
  • 3: 6 or 7
  • 2: 4 or 5
  • 1: 3 or fewer, or paste-only with a tight cap

6.Subject Coverage

The results from the accuracy test are broken out into the three subject groups (maths and science; humanities; general knowledge), each out of 20, so a tool that is strong at maths but invents history, or vice versa, is caught. The grade is set by the weakest subject, not the average, because a student studying the subject the tool is worst at gains nothing from its strength elsewhere. The per-subject table is published, the strongest and weakest subjects are named, and any subject the tool openly says it does not cover (a maths-only solver, a language-only tutor) is noted and that subject left out of its score rather than counted as a wall of misses.

  • 5: every subject group at 90% or more
  • 4: every group at 80% or more
  • 3: every group at 65% or more, or one group below with the rest strong
  • 2: one subject group under half correct
  • 1: two or more groups under half correct

7.Citations & Trustworthiness

On the factual and research questions, and on questions asked about the uploaded documents, whatever sources, page references or citations the tool gives are checked one by one: does the source exist, does it say what the tool claims, and, for document questions, does the answer stay inside the material the student supplied. A fabricated reference, a real source that does not support the point, a page number that is wrong, or an answer drawn from outside the uploaded document when the tool claimed to use it, is counted and listed. Whether the tool shows where an answer came from at all, and whether it says plainly when it does not know, are reported, because a study tool a student cannot check back is one they have to trust blind.

  • 5: citations real and supporting, answers stay on the supplied source
  • 4: occasional weak reference, nothing fabricated
  • 3: sources sometimes missing or loose but no invented citations
  • 2: some fabricated or non-supporting citations
  • 1: citations routinely invented, or answers made up and dressed as sourced

8.Mobile & App Experience

Each point is checked on a phone on the test date: a working mobile app or a site that renders properly on a phone; sign-in and the core study flow usable one-handed; uploading or photographing notes from the phone; generating and reviewing flashcards on the phone; taking a quiz on the phone; studying offline or with saved material when there is no connection; notifications or a study reminder or streak; syncing the same material between phone and desktop; readable text and controls without pinch-zoom; and no broken, overlapping or cut-off layout on a normal phone screen. A point counts only when it works on the account used, and each is screenshotted on the phone.

  • 5: all ten points
  • 4: 8 or 9
  • 3: 6 or 7
  • 2: 4 or 5
  • 1: 3 or fewer, or no usable mobile experience

9.Free-Tier Limits

On the free tier I study until the tool stops me and record the real limits: how many questions, uploads, flashcard sets or quizzes a day, whether an account or a card is needed, whether the free tier runs the same model and features as the paid one, and which of the core jobs (answers, explanations, flashcards, quizzes, uploads, the mobile app) are free and which are walled off. A tool whose free tier is a demo that does nothing useful without paying is marked down here; a tool a student could genuinely study on for free scores well, with the exact limits stated and dated. Credit or trial-only access is recorded as what a typical day's studying actually costs once the free allowance runs out.

  • 5: a free tier a student can study on day to day, core jobs included
  • 4: a usable free tier with one core job walled off
  • 3: a tight free tier, enough to try but not to rely on
  • 2: a demo-only free tier, or a card required to start
  • 1: nothing useful without paying

10.Pricing & Value

Every plan is read from the checkout, not the marketing page, with the date: the monthly cost billed yearly, any separate student or education price and what it needs to qualify, what the paid plan lifts or adds over the free tier, and any credit or per-use metering. The cost of a typical month's studying is worked out in dollars on the cheapest plan that removes the free cap, and any auto-renewing trial or hard-to-cancel billing is flagged, since these tools are sold to students on tight budgets. A plan that unlocks the real study features cheaply scores well; one that charges a lot for little over the free tier, or hides the price, scores low.

  • 5: under $10 a month (or a real student price) that unlocks the full study features
  • 4: $10 to $14.99 a month with clear value
  • 3: $15 to $19.99, or value only on the yearly plan
  • 2: $20 or more, or thin value for the price
  • 1: poor value, auto-renew traps or unclear pricing

Documented but not graded

An AI tutor or chat, an essay or writing assistant, a plagiarism or AI-detection check, a note-taker or transcription, spaced-repetition scheduling, a shared flashcard or study-set library, classroom and teacher dashboards, LMS and Google Classroom links, a browser extension, exam-prep and past-paper banks, language and accessibility features (text-to-speech, translation), and an API are checked and described in every review. When they work well they count in the written assessment, but they never add points to the overall grade.

Rules that apply to every test

  • Real use of the product on a real account, never vendor accuracy claims, app-store ratings or promotional descriptions.
  • Identical questions, documents, source material, upload and sample sizes for every study tool in the category, with the first answer counted every time and the best plan the account offers.
  • Every question has an answer key worked out and checked before any tool is opened, so an answer can be marked right, wrong or invented rather than judged by impression.
  • Every grade has evidence: the question, the answer, the generated cards or quiz, a count, a measurement, a screenshot or a table, in the full test results.
  • A grade changes only when a retest produces new evidence; the review records the date, the model or app version and the reason, because these tools change their models and limits often.

Questions about a specific grade go to the contact page. Makers can submit an AI study tool for testing. The tested ranking is on the AI study tools page.