RankDude

How I test AI SEO automation

Nine tests, the same for every tool, each graded from 1 to 5 on a fixed scale. The overall grade is their average plus my verdict, which can move it by up to one point. Every test is run by hand on an account I use, against a real website I control, and the counts, cross-checks, screenshots and exports behind each grade go in the full test results published with the review.

Written by Miles Foster, Owner and tester, RankDude.

Each tool is pointed at the same test site: a 60-page WordPress site for a fictional plumbing company, with a year of Google Search Console history, 25 faults I planted on purpose (missing titles, duplicate pages, broken links, slow images, thin pages, missing schema, orphan pages) and a list of 30 keywords I already know the true rankings for. I connect Search Console and the CMS where the tool allows it, let it audit, research, write, fix and report, then check every claim it makes against the site itself and against Search Console. Nothing is graded from a feature list, a case study or a demo.

Each review states how the account was obtained: bought at the public price, used on a free tier, or provided by the vendor. A vendor-provided account never changes a grade, and a vendor never sees a review before it is published.

How I rate

Every tool gets one score out of 5.

First, I run the same 9 tests on every tool and give each one a grade from 1 to 5. The average of the graded tests is the starting point; a test that cannot be graded is left out of it, and the review says how many tests the score is based on.

Then I add my verdict: what the tests cannot measure, such as how the tool holds up after the test weeks, how billing and cancelling work in practice, and how it compares with the others in the category. My verdict can move the score up or down by one point at most.

Every review shows how much my verdict changed the score, and why, in its scores table.

What a grade means

5Excellent
4Good
3Average
2Below average
1Poor

The final score is the average of the graded tests (nine when every test can be graded), to one decimal place, plus my verdict, which can move it by up to one point. If a test couldn't be finished properly it gets no grade, and the review says how many tests the score is based on. A tool that only does part of the job (a writer with no audit, a tracker with no writer) is graded on what it offers, and the review says which tests it could not take.

The 9 tests

1.Setup to First Recommendations

The clock starts at sign-up and stops when the tool has connected to the test site (Search Console, a crawl, a CMS plugin or a script, whichever it uses), finished its first audit and shown a prioritised list of actions. Verification steps, plugin installs and any waiting for a crawl count. Every connection that failed or needed support is logged.

  • 5: under 30 minutes
  • 4: 30 to 59 minutes
  • 3: 1 to 3 hours
  • 2: 3 to 24 hours
  • 1: over 24 hours, or the site could not be connected

2.Keyword Research

I ask the tool for keyword ideas for five seed topics. A fixed list of 25 real opportunities (queries the site already gets impressions for without ranking in the top ten) is the answer sheet; I count how many the tool finds. Its monthly volume for 20 keywords is compared with twelve months of Search Console impressions, and clustering is checked on 100 queries I have grouped by hand. Invented keywords and nonsense clusters are counted.

  • 5: 20 or more of 25 found, volumes within 30% on at least 15 of 20 keywords
  • 4: 15 to 19 found
  • 3: 10 to 14 found, or volumes out by more than 30% on most keywords
  • 2: 5 to 9 found
  • 1: fewer than 5 found, or invented keywords

3.Site Audit Accuracy

The audit report is checked line by line. Each of the 25 planted faults either appears or does not. Every reported issue that is not real (a false alarm, or a fault the tool itself caused) is counted. The tool's priority order is compared with mine: the five faults that actually block indexing or hurt the biggest pages should be near the top.

  • 5: 23 to 25 found, 0 to 2 false alarms
  • 4: 19 to 22 found, or 3 to 5 false alarms
  • 3: 13 to 18 found, or 6 to 10 false alarms
  • 2: 7 to 12 found, or more than 10 false alarms
  • 1: 6 or fewer found, or the audit never completed

4.On-Page & Technical Fixes

The twelve: rewrite a title and meta description, add a missing H1, fix a broken internal link, add three internal links to an orphan page, add schema to a service page, write alt text for ten images, set a canonical, redirect a dead page, compress or lazy-load the images on one page, add an FAQ block, fix a duplicate page, and update a page's copy from a brief. Each counts only when the change is live on the site (I check the source) and did not break anything else. I record whether the change needed my approval first, and whether it can be rolled back.

  • 5: 10 to 12 applied and live, nothing broken, rollback available
  • 4: 7 to 9 applied, or one change broke something
  • 3: 4 to 6 applied, or suggestions only, done by me from clear instructions
  • 2: 1 to 3 applied, or two or more changes broke something
  • 1: none applied, or a change damaged the site without a rollback

5.Writing Quality & Facts

Five briefs, each with ten facts about the plumbing company and a target keyword, produce five articles in one pass with no hand edits. Each fact is checked. Invented content (prices, statistics, quotes, certifications, places) is counted. Each article is run through a plagiarism check and read against the top three ranking pages for copying of structure or wording. An article counts as publishable when I would put it live after ten minutes of editing. The tool's own optimisation score, if it has one, is recorded alongside.

  • 5: no wrong facts, nothing invented, five of five publishable
  • 4: one wrong or invented item, four publishable
  • 3: two or three wrong or invented items, or three publishable
  • 2: four to six wrong or invented items, or two publishable
  • 1: seven or more wrong or invented items, copied passages, or none publishable

6.Data Accuracy

For 30 keywords I check the site's real Google position by hand in a clean browser on the same day and compare the tool's reported position. Organic traffic for the last month is compared with Search Console clicks. Referring domains are compared with a hand-built list of the site's known links. Any figure the tool shows without a source, or a date, is noted.

  • 5: 27 to 30 keyword positions within one place, traffic within 20%
  • 4: 22 to 26 positions within one place
  • 3: 15 to 21, or traffic out by 20% to 50%
  • 2: 8 to 14, or traffic out by more than 50%
  • 1: 7 or fewer, or figures that could not be reconciled at all

7.Rank Tracking & Reporting

The 30 keywords are tracked for four weeks. I count missed days, keywords with no data and positions that disagree with my weekly hand check. I then build one report that shows what moved and why, with a date range, Search Console data alongside the tool's own, and export it as PDF or a shared link. Automated alerts for a ranking drop are triggered on purpose by removing a page, and the time to the alert is recorded.

  • 5: daily positions on all 30, no missed days, drop alert within 48 hours, exportable report
  • 4: one or two keywords with gaps, or alert within a week
  • 3: weekly positions only, or report with no export
  • 2: more than five keywords with gaps, or no alerts
  • 1: no rank tracking, or positions that were wrong against the hand check more often than right

8.AI Search Visibility

Twenty prompts about plumbing in the site's town are run by hand in Google (AI Overviews and AI Mode), ChatGPT and Gemini on the same day, and I note where the test site is cited. The tool's visibility report for the same prompts is compared with those checks. I record which assistants it covers, how often it refreshes, and whether it gives a source for each citation it reports.

  • 5: covers Google AI, ChatGPT and Gemini, matches 18 to 20 of 20 prompts
  • 4: covers two of the three, or matches 14 to 17
  • 3: covers one, or matches 10 to 13
  • 2: matches fewer than 10, or data more than a month old
  • 1: no AI visibility tracking, or reported citations that did not exist

9.Pricing & Value

Every plan is read from the checkout, not the marketing page. The figure graded is the yearly cost of the cheapest plan that connects one site, tracks 30 keywords and includes enough AI credits to run every test above without topping up. Per-site, per-keyword and per-article charges, the renewal price against the first-year price and the payment methods seen at checkout are listed with it.

  • 5: under $600 a year, no credit cap hit, same price at renewal
  • 4: $600 to $1,199 a year
  • 3: $1,200 to $2,999, or renewal more than 50% higher
  • 2: $3,000 or more, or credits ran out during the tests
  • 1: poor value or costs that were not shown before paying

Documented but not graded

Backlink prospecting, local listings, competitor gap reports, content calendars, white-label options and the tool's own AI assistant are checked and described in every review. When they work well they count in the written assessment, but they never add points to the overall grade.

Rules that apply to every test

  • Real use of the product on a real account and a real website, never promotional descriptions or case studies.
  • Identical test site, planted faults, keyword lists, briefs, tests, thresholds and sample sizes for every tool in the category.
  • Authorised accounts only; no password sharing, no bypassing of plan limits or crawl limits, and no changes to any site I do not own.
  • Every grade has evidence: a count, a cross-check, a screenshot, an export or a table, in the full test results.
  • A grade changes only when a retest produces new evidence; the review records the date and the reason.

Questions about a specific grade go to the contact page. Makers can submit an AI SEO automation tool for testing. No AI SEO automation tool has been through these tests yet; every one I follow is listed on the AI SEO automation page, and reviews appear there as tests finish.