RankDude

How I test AI sales agents

Nine tests, the same for every agent, each graded from 1 to 5 on a fixed scale. The overall grade is their average plus my verdict, which can move it by up to one point. Every test is run by hand on an account I use, with a real outbound campaign to real inboxes I control, and the lead lists, emails, reply logs, CRM records and cost sheets behind each grade go in the full test results published with the review.

Written by Miles Foster, Owner and tester, RankDude.

Each agent runs the same campaign for the same fictional business: a software company selling a scheduling tool to dental practices. It gets the same written profile of the ideal customer, the same offer, the same brand voice guide, a fresh sending domain, a HubSpot CRM with 200 existing contacts, and a list of 300 target companies that I built and verified by hand, of which 100 are seeded with inboxes I control. Those 100 inboxes reply on a fixed schedule: 30 ask a question, 20 say not now, 15 ask to be removed, 10 say yes to a meeting, 5 reply out of office, and 20 never answer. The agent prospects and enriches the whole list, but sends only to the 100 seeded inboxes: the other 200 companies sit on a suppression list, so no real practice is emailed. Over four weeks it writes, sends, handles the replies and books the meetings. Each outcome has an answer key. Nothing is graded from a vendor's reply-rate claim or a case study.

Each review states how the account was obtained: bought at the public price, used on a free tier, or provided by the vendor. A vendor-provided account never changes a grade, and a vendor never sees a review before it is published.

How I rate

Every tool gets one score out of 5.

First, I run the same 9 tests on every tool and give each one a grade from 1 to 5. The average of the graded tests is the starting point; a test that cannot be graded is left out of it, and the review says how many tests the score is based on.

Then I add my verdict: what the tests cannot measure, such as how the tool holds up after the test weeks, how billing and cancelling work in practice, and how it compares with the others in the category. My verdict can move the score up or down by one point at most.

Every review shows how much my verdict changed the score, and why, in its scores table.

What a grade means

5Excellent
4Good
3Average
2Below average
1Poor

The final score is the average of the graded tests (nine when every test can be graded), to one decimal place, plus my verdict, which can move it by up to one point. If a test couldn't be finished properly it gets no grade, and the review says how many tests the score is based on. Some tests on a free plan cannot send, connect a CRM or bring their own list; I test them again on a paid plan when I can.

The 9 tests

1.Lead Quality

The agent is asked for 200 leads. Each lead is checked on five points: the company matches the profile (a dental practice of the right size in the right country), the person holds the role asked for, the email address is valid and belongs to that person, the company is still trading, and the lead is not already in the CRM. A lead counts when all five hold. Bounces, wrong people and duplicates are reported separately.

  • 5: 180 to 200
  • 4: 160 to 179
  • 3: 130 to 159
  • 2: 90 to 129
  • 1: fewer than 90

2.Writing & Personalisation

Forty first-touch emails are sampled at random. Each is scored on: under 120 words, one clear ask, a subject line under eight words, a personal line that is true and specific to the recipient, no placeholder text, the offer stated correctly, the brand voice guide followed, no spam-trigger formatting, a working unsubscribe line, and no false claim about the recipient, their company or my product. Invented facts are counted separately and cap the grade.

  • 5: median 9 or 10, nothing invented
  • 4: median 8, at most one invented item
  • 3: median 6 or 7, or two or three invented items
  • 2: median 4 or 5, or placeholder text sent
  • 1: median 3 or fewer, or more than three invented items

3.Reply Handling

Each reply has a correct outcome: answer the question accurately, set a follow-up for the not-now replies at the date given, remove the opt-outs within one hour and never email them again, book the yeses, pause for out-of-office until the return date, and escalate anything it cannot answer to me. I count correct outcomes and list every wrong one, with an email sent to an opt-out counting as two failures.

  • 5: 72 to 80
  • 4: 64 to 71
  • 3: 52 to 63
  • 2: 36 to 51, or one opt-out emailed again
  • 1: fewer than 36, or two or more opt-outs emailed again

4.Meetings Booked

Each yes gives a time preference and a time zone. A meeting counts when it appears in the connected calendar at a time that matches the preference, in the right zone, with the right person, a working video link and a confirmation email to both sides, with no step done by me. A meeting booked at the wrong time or with the wrong person is a failure, as is one the agent says it booked but did not.

  • 5: 9 or 10
  • 4: 7 or 8
  • 3: 5 or 6
  • 2: 3 or 4
  • 1: 2 or fewer, or no booking possible

5.CRM Accuracy

For 100 contacts, 50 seeded and 50 not, I check that the contact exists once, the fields the agent filled are correct, every email sent and reply received is logged, the stage matches the real outcome (replied, meeting booked, opted out, no reply), opt-outs are marked do-not-contact, and nothing in the 200 pre-existing contacts was overwritten or duplicated. A record counts when all of that holds.

  • 5: 95 to 100
  • 4: 85 to 94
  • 3: 70 to 84
  • 2: 50 to 69, or an existing record overwritten
  • 1: fewer than 50, or no CRM connection

6.Deliverability & Domain Care

The 100 seeded inboxes span Gmail, Outlook and two other providers; I count how many first emails reached the inbox rather than spam or nowhere. I check whether the agent set up or verified SPF, DKIM and DMARC, warmed the mailboxes, kept daily volume within the limits it stated, and paused when I planted 15 bouncing addresses in the list. The domain's reputation is checked before and after on the same tool.

  • 5: 95% or more inboxed, all checks done, paused on bounces
  • 4: 85% to 94%
  • 3: 70% to 84%, or one check missing
  • 2: 50% to 69%, or no pause on bounces
  • 1: under 50%, or the domain was blocklisted

7.Approval & Compliance

I run the first week in approval mode, if there is one, and count emails that went out without my approval. I check that every email carries a physical address and an unsubscribe method, that the agent will not email a suppression list of 20 addresses I upload, that it respects a daily cap I set, and that LinkedIn or phone actions, where offered, stay within those platforms' rules. Then I check the 15 opt-outs again at the end of week four.

  • 5: all checks pass, nothing sent without approval
  • 4: one check missed
  • 3: two checks missed, or no approval mode
  • 2: an email sent without approval, or a suppressed address emailed
  • 1: no unsubscribe method, or opt-outs ignored

8.Setup & Channels

The clock starts at sign-up and stops when the first approved email to a seeded inbox is sent. Then seven connections are tried for real: Gmail, Outlook, HubSpot, a LinkedIn action such as a connection request, a calendar, my own uploaded lead list, and a data source the agent enriches from. Each counts when it does its job in the campaign; one that needs the vendor's team to set up counts as half, and a half is rounded down in the total.

  • 5: under 4 hours, all seven
  • 4: 4 to 24 hours, or six
  • 3: 1 to 3 days, or four or five
  • 2: more than 3 days, or two or three
  • 1: could not send, or one or none

9.Pricing & Cost per Meeting

Every plan is read from the checkout or the written quote, not the marketing page. The campaign runs on the cheapest plan that supports the connections it needs, and every bill is collected: platform fee, credits for leads or emails, data costs, extra mailboxes and domains, and any model fee. The figure graded is the total divided by the meetings correctly booked, with the cost per lead contacted reported alongside. I note minimums, contract length, the renewal price against the first-month price and the payment methods seen at checkout.

  • 5: under $60 a meeting, all costs known in advance
  • 4: $60 to $119
  • 3: $120 to $249, or a cost that only appeared afterwards
  • 2: $250 to $499, or an annual contract required
  • 1: $500 or more, or pricing only through a sales call with no figure given

Documented but not graded

Phone and voice outreach, LinkedIn content tools, intent and signal data, call recording and coaching, team dashboards and multilingual outreach are checked and described in every review. When they work well they count in the written assessment, but they never add points to the overall grade.

Rules that apply to every test

  • Real use of the product on a real account and a real campaign to inboxes I control, never vendor reply-rate claims or case studies.
  • Identical business, profile, offer, lists, seeded replies, tests, thresholds and sample sizes for every agent in the category.
  • Authorised accounts only; no password sharing, no bypassing of plan or platform limits, and outreach only to addresses and profiles I own or have permission to use.
  • Every grade has evidence: a lead list, an email, a reply log, a CRM export, a calendar record or a table, in the full test results.
  • A grade changes only when a retest produces new evidence; the review records the date and the reason.

Questions about a specific grade go to the contact page. Makers can submit an AI sales agent for testing. No AI sales agent has been through these tests yet; every one I follow is listed on the AI sales agents page, and reviews appear there as tests finish.