How I test AI sales automation

Nine tests, the same for every tool, each graded from 1 to 5 on a fixed scale. The overall grade is their average plus my verdict, which can move it by up to one point. Every test is run by hand on an account I use, the outreach goes to inboxes and contacts I control, and the timings, seed-inbox results, transcripts and checklists behind each grade go in the full test results published with the review.

Written by Miles Foster, Owner and tester, RankDude.

The RankDude mascot holding a laptop with a check on its screen

Each tool gets the same job: sell a fictional bookkeeping service to small accountancy firms. It gets the same ideal customer profile, the same two sending mailboxes on a test domain, a test HubSpot CRM and a calendar. It must find leads, write a sequence, send it, handle the replies and keep the CRM up to date. The replies come from 20 test contacts I control, who answer in fixed ways. Nothing is graded from a feature list or a case study.

Each review states how the account was obtained: bought at the public price, used on a free tier, or provided by the vendor. A vendor-provided account never changes a grade, and a vendor never sees a review before it is published.

How I rate

Every tool gets one score out of 5.

First, I run the same 9 tests on every tool and give each one a grade from 1 to 5. The average of the graded tests is the starting point; a test that cannot be graded is left out of it, and the review says how many tests the score is based on.

Then I add my verdict: what the tests cannot measure, such as how the tool holds up after the test weeks, how billing and cancelling work in practice, and how it compares with the others in the category. My verdict can move the score up or down by one point at most.

Every review shows how much my verdict changed the score, and why, in its scores table.

What a grade means

5	Excellent
4	Good
3	Average
2	Below average
1	Poor

The final score is the average of the graded tests (nine when every test can be graded), to one decimal place, plus my verdict, which can move it by up to one point. If a test couldn't be finished properly it gets no grade, and the review says how many tests the score is based on. Some tests on a free plan can't connect a CRM or buy contact data; I test them again on a paid plan when I can.

The 9 tests

1.Setup to First Sequence

The clock starts at account creation and stops when the first email of the sequence has left a connected mailbox, with both mailboxes authenticated, the HubSpot test CRM connected and the calendar linked for booking. Warm-up waits the tool imposes are reported but not counted. I note every step that needed support or a developer.

5: under 2 hours, no help
4: 2 to 4 hours
3: 4 to 8 hours, or one step needed support
2: 1 to 3 days
1: over 3 days, or the CRM could not be connected

2.Lead & Data Accuracy

The tool builds a list of 100 leads for the profile: owners or partners at accountancy firms with 2 to 20 staff in the UK. I check 50 at random against the firm's website and a professional profile: right role, right firm size, right country, and the email address verified without a bounce (checked with a verification service, not by sending). A lead counts only when all four are right.

5: 45 to 50 right
4: 40 to 44
3: 30 to 39
2: 20 to 29
1: under 20, or no lead finding

3.Outreach Writing

The tool writes a first email for 30 of the verified leads. Each email is checked against the same rules: the right name and firm, no invented facts about the firm, no invented claims about my service, the offer from the brief present, under 120 words, a subject line under eight words, a single clear ask, no broken merge fields, no spam-trigger phrases from a fixed list, and a working unsubscribe or opt-out line. An email passes only when all ten rules hold.

5: 27 to 30 pass
4: 23 to 26
3: 17 to 22
2: 10 to 16
1: under 10, or no AI writing

4.Sequence & Workflow

The sequence is the same for every tool: email one, wait three days, email two only if no reply, a LinkedIn or call task created for the rep, a branch on a link click, a third email with a different template, stop the sequence the moment a reply or a booking arrives, and write the outcome to the CRM. I run it with the 20 test contacts and check that every step fires at the right time and that nobody gets an email after replying.

5: all eight steps right
4: one step missing or late
3: two steps missing, or one contact emailed after replying
2: three or more steps missing
1: the sequence could not be built

5.Deliverability & Sending

The seed list has 20 inboxes across Gmail, Outlook, Yahoo and iCloud. For each of three sends I record where the email landed: inbox, promotions, spam or missing. I check that SPF, DKIM and DMARC pass in the headers, whether the tool offers warm-up and sending limits per mailbox, whether it rotated between the two mailboxes as set, and whether it kept to the daily limit I set.

5: 90% or more in the inbox, all checks pass
4: 80 to 89%
3: 65 to 79%, or one check fails
2: 50 to 64%, or a limit ignored
1: under 50%, or authentication cannot be set up

6.Reply Handling & Booking

The 20 test contacts reply in fixed ways: interested and asking for a time, interested but later, a question about price, the wrong person with a referral, an out-of-office, a hard no, an unsubscribe, and an angry reply. I check that each reply is classified correctly, that the tool's own reply (where it writes one) is accurate and polite, that meetings land in the calendar at the agreed time with an invitation sent, that the sequence stops, and that unsubscribes and hard noes are never contacted again.

5: all twenty right, all meetings booked
4: 17 to 19
3: 13 to 16
2: 8 to 12, or one unsubscribe contacted again
1: 7 or fewer, or no reply handling

7.CRM Sync

I check twenty things in the test CRM: ten activities that should appear (emails sent, opens, replies, a booked meeting, a task, a sequence status, a lead status change) and ten changes I make in the CRM that should flow back (a new owner, a do-not-contact flag, a changed email, a deal stage, a deleted contact). Each counts when it appears correctly within an hour without duplicates.

5: all twenty
4: 17 to 19
3: 13 to 16
2: 8 to 12, or duplicates created
1: 7 or fewer, or no CRM sync

8.Reporting

I keep my own log of every send, open, reply and meeting. The eight questions: emails sent per mailbox, reply rate per step, positive replies, meetings booked, bounces, unsubscribes, the best-performing template, and a CSV export of every contact's status. An answer counts when the figure matches my log exactly for counts and within 2 points for rates.

5: all eight right
4: 6 or 7
3: 4 or 5
2: 2 or 3
1: 1 or none

9.Pricing & Value

Every plan is read from the checkout, not the marketing page. The figure graded is the yearly cost of the cheapest plan that covers one seat, two sending mailboxes, the CRM connection, AI writing and 1,000 new contacts a month including any data credits. Per-meeting or per-outcome charges, extra mailbox fees, minimum terms, the renewal price against the first-year price and the payment methods seen at checkout are listed.

5: under $1,200 a year, everything included, same price at renewal
4: $1,200 to $2,399
3: $2,400 to $4,799, or data credits cost extra
2: $4,800 or more, or quote-only pricing
1: poor value or hidden costs

Documented but not graded

Calling and voice agents, LinkedIn automation, intent signals, call recording and coaching, team features and support are checked and described in every review. When they work well they count in the written assessment, but they never add points to the overall grade.

Rules that apply to every test

Real use of the product on a real account, real mailboxes and real test contacts, never promotional descriptions or case studies.
Identical profile, briefs, tests, thresholds and sample sizes for every tool in the category.
Authorised accounts only; no password sharing, no bypassing of plan limits or bot checks, and outreach sent only to contacts and inboxes I control.
Every grade has evidence: a measurement, a seed-inbox result, a transcript or a checklist, in the full test results.
A grade changes only when a retest produces new evidence; the review records the date and the reason.

Questions about a specific grade go to the contact page. Makers can submit an AI sales automation tool for testing. No AI sales automation tool has been through these tests yet; every one I follow is listed on the AI sales automation page, and reviews appear there as tests finish.