RankDude

How I test AI receptionists

Nine tests, the same for every service, each graded from 1 to 5 on a fixed scale. The overall grade is their average plus my verdict, which can move it by up to one point. Every test is run by hand on an account I use, with scripted calls I place myself, and the recordings, transcripts, timings and screenshots behind each grade go in the full test results published with the review.

Tested and written by Miles Foster, Owner and tester, RankDude.

Each receptionist is set up for the same imaginary business: a three-person plumbing firm with a published price list, opening hours, a service area and a calendar with gaps in it. It gets the same 20 facts, the same booking rules and the same instructions about who to transfer to. Then I phone it: eight scripted calls, seven in business hours and one after closing, each built to catch one kind of failure: a normal caller with four questions, a question whose answer depends on the rules, a price trap pushed four ways, a booking, a change and then a cancellation of that booking, a message with the name spelled out, and an emergency that needs a person. Each call gets a pass or a fail against the failure it hunts, and the nine tests are graded from the same recordings.

The calls go to the service's phone number when the account has one, or over the service's own browser test line when it does not, and the review says which. A test line that cannot transfer calls or send texts limits those grades, and the review says that too. Every call is recorded, and nothing is graded from a feature list or a demo video. Reviews published before 24 October 2026 ran an earlier 25-call plan; their pages say so.

Each review states how the account was obtained: bought at the public price, used on a free tier, or provided by the vendor. A vendor-provided account never changes a grade, and a vendor never sees a review before it is published.

How I rate

Every tool gets one score out of 5.

First, I run the same 9 tests on every tool and give each one a grade from 1 to 5. The average of the graded tests is the starting point; a test that cannot be graded is left out of it, and the review says how many tests the score is based on.

Then I add my verdict: what the tests cannot measure, such as how the tool holds up after the test weeks, how billing and cancelling work in practice, and how it compares with the others in the category. My verdict can move the score up or down by one point at most.

Every review shows how much my verdict changed the score, and why, in its scores table.

What a grade means

5Excellent
4Good
3Average
2Below average
1Poor

The final score is the average of the graded tests (nine when every test can be graded), to one decimal place, plus my verdict, which can move it by up to one point. If a test couldn't be finished properly it gets no grade, and the review says how many tests the score is based on. Some services cannot transfer calls or connect a calendar on their cheapest plan; I test them again on a higher plan when I can.

The 9 tests

1.Pick-up & Response Speed

Every call is timed from the first ring, or from the moment the test line connects, to the greeting, and checked for a missed or dropped pick-up. The gap between the end of my sentence and the start of the receptionist's reply is measured on every turn of every call, from the recording. The grade is the pick-up rate and the median reply gap together; the longest gap is reported alongside.

  • 5: every call answered, median reply gap under 1 second
  • 4: every call answered, median gap 1 to 1.5 seconds
  • 3: one call missed or dropped, or median gap 1.5 to 2.5 seconds
  • 2: two calls missed, or median gap 2.5 to 4 seconds
  • 1: three or more calls missed, or median gap over 4 seconds

2.Voice & Conversation

One 30-second sample from each of three calls is rated by me on naturalness, clarity, pacing and intonation, each on a 1 to 5 line. Robotic pauses, repeated phrases and talking over the caller are counted as faults from the recordings. When a review can run it, an interruption test adds to this: I interrupt mid-sentence and count whether the receptionist stops, listens and answers the new question.

  • 5: my rating 4.5 or more, no talking over the caller
  • 4: rating 4 to 4.4
  • 3: rating 3 to 3.9, or the receptionist talks over the caller on one call
  • 2: rating 2 to 2.9
  • 1: rating under 2, or the receptionist talks over the caller on most calls

3.Answer Accuracy

The business profile holds 20 facts: prices, hours, service area, what the firm does and does not do, payment methods, the booking rules. The calls ask for the facts a real caller asks: the hours, the area, the call-out fee, the hourly rate, a listed price, what counts as an emergency, the arrival windows, the free-estimate rule, the senior discount and the cancellation rule. Each fact stated is checked against the profile. Invented answers (a price, a service or a promise that is not in the profile) are counted separately and weigh more than a plain 'I don't know'.

  • 5: every stated fact right, nothing invented
  • 4: one fact wrong, or one invented answer
  • 3: two facts wrong, or two invented answers
  • 2: three or four facts wrong, or three invented answers
  • 1: five or more wrong, or four or more invented answers

4.Appointment Booking

The calendar has fixed gaps and one deliberate clash: a day the plumber is off. One call books a job, a second moves it to the day off and takes the alternative offered, a third cancels it. After each call I open the calendar and check the date, time, duration, customer name, phone number and job description. A booking in a slot that was already taken, or at a time the business is closed, counts as wrong. A change or a cancellation the receptionist reports as done that is not on the calendar counts as wrong and is reported first. If the service only takes a request for a person to confirm and tells the caller so, that is graded as a 3 at best.

  • 5: all three correct, no clash
  • 4: the booking and one of the two changes right, with every detail correct
  • 3: the booking right and the changes are requests a person has to confirm, with the caller told so
  • 2: one of three right, a double booking, or a change or cancellation reported as done that was not made
  • 1: no booking at all

5.Messages & Lead Capture

One caller leaves a new enquiry with the name spelled out, a number, an address, the reason and a best time to call; the after-hours emergency caller leaves a message when nobody answers the transfer. I check the message, ticket or lead record for every field, and time how long it takes to arrive by text, email or in the dashboard. A wrong phone number counts as a failed message, because the call-back never happens.

  • 5: both complete and correct, delivered within 2 minutes
  • 4: one field missing, or delivery within 10 minutes
  • 3: one message complete, or delivery within an hour
  • 2: a wrong phone number in either message
  • 1: neither message usable, or messages only visible by logging in

6.Transfers & Human Handoff

The profile names two people and the reasons each should get a call. The daytime emergency asks for the owner; I count whether the right person is named in the first reply, whether the call reaches the right phone without an intake form first, and whether the person hears a summary. The after-hours emergency goes unanswered; I check what the caller is told, whether a message with name, number and reason is taken, and whether a call-back time is promised that the business never gave. On a browser test line that cannot transfer, the grade rests on what the receptionist says it would do, and the review says so.

  • 5: the right person at once on both, with a summary, and the failed transfer falls back to a full message
  • 4: the right person at once, but no summary is passed or the fallback misses a detail
  • 3: a person reached only after questions the caller should not have to answer first
  • 2: the caller is cut off, or left without a message, when nobody answers
  • 1: no path to a person at all

7.Pressure & Traps

The trap call asks for a price that is not on the list, twice; quotes a competitor's hourly rate and asks the receptionist to match it; and asks for the call-out fee to be waived. The no-hot-water call pushes for a same-day visit that the rules do not allow. The normal caller asks whether they are talking to a real person or a computer. Each push is marked held or conceded from the recording, and I note anything the receptionist agreed to that the business never offers.

  • 5: all six held, nothing wrongly promised, the real-person question answered honestly
  • 4: five held
  • 3: four held, or one wrong promise
  • 2: two or three held, or the receptionist loops or hangs up on a real caller
  • 1: one or none held

8.Setup & Integrations

The clock starts at sign-up and stops when a forwarded call to the business number is answered correctly, my own fixes included. Then each point is tried, not read from a feature list: call forwarding from an existing number, a new number, Google Calendar, Outlook calendar, a CRM or job app (whichever the service offers), Zapier or a webhook, a text to the caller after the call, call recordings, transcripts, and after-hours rules that differ from daytime rules. Where no phone number is used, the clock stops at the first answered test call, the forwarding point is marked not done, and the test is not graded until a real number is used.

  • 5: live within 30 minutes, 9 or 10 points working
  • 4: live within 2 hours, 7 or 8 points
  • 3: live the same day, 5 or 6 points
  • 2: live within a week, 3 or 4 points, or setup needed a sales call
  • 1: not live within a week, or 2 or fewer points

9.Pricing & Value

Every plan is read from the checkout or order page, not the marketing page. The figure graded is the monthly cost of handling 200 calls averaging two minutes, on the cheapest plan that includes booking and transfers, overage and any per-call fees included. I also record the billing unit (per minute, per call, per caller or flat), the contract length, the trial, whether a card is needed to try it, what the terms say about recordings and caller data, and whether a HIPAA agreement is offered. Renewal and introductory prices are listed separately.

  • 5: under $100 a month, no overage surprise, monthly contract
  • 4: $100 to $199 a month
  • 3: $200 to $349, or a yearly contract
  • 2: $350 or more, or the price is only available from a sales call
  • 1: poor value or hidden costs

Documented but not graded

Languages beyond English, outbound call-backs, text and web chat handling, call analytics, a mobile app and the quality of the dashboard are checked and described in every review. When they work well they count in the written assessment, but they never add points to the overall grade.

Rules that apply to every test

  • Real calls on a real account: to the service's phone number when the account has one, or over its own test line when it does not, and the review says which. Never promotional descriptions or demo recordings.
  • Identical business profile, call scripts, thresholds and sample sizes for every service in the category.
  • Authorised accounts only; no password sharing, no bypassing of plan limits, and every test call made by me or by someone who knows it is recorded.
  • Every grade has evidence: a recording, a transcript, a timing, a screenshot or a table, in the full test results.
  • A grade changes only when a retest produces new evidence; the review records the date and the reason.

Questions about a specific grade go to the contact page. Makers can submit an AI receptionist for testing. The tested ranking is on the AI receptionists page.