How I test AI front desk tools

Nine tests, the same for every tool, each graded from 1 to 5 on a fixed scale. The overall grade is their average plus my verdict, which can move it by up to one point. Every test is run by hand on an account I use, with real calls, texts and web chats to a real business profile over a working week, and the recordings, screenshots, timings and records behind each grade go in the full test results published with the review.

Written by Miles Foster, Owner and tester, RankDude.

The RankDude mascot holding a laptop with a check on its screen

Each tool is set up as the front desk of the same imaginary business: a two-chair dental practice with a published fee list, opening hours, two clinicians with different schedules and a calendar that already has gaps and clashes in it. It gets the same 20 facts, the same booking rules and the same intake questions. Then it gets a week of front-desk work: 40 phone calls, 20 text conversations and 20 web chats that book, move, cancel, ask, complain and check in, plus ten appointments that need reminders. A tool built only for hotels or salons gets the same week recast for its trade, a 20-room hotel or a three-chair salon with the same number of facts, rules, contacts and reminders (guests or clients instead of patients), and the review says which profile it used; grades compare within a profile. Every call is recorded with the caller's knowledge, every text and chat is saved, and nothing is graded from a feature list or a demo.

Each review states how the account was obtained: bought at the public price, used on a free tier, or provided by the vendor. A vendor-provided account never changes a grade, and a vendor never sees a review before it is published.

How I rate

Every tool gets one score out of 5.

First, I run the same 9 tests on every tool and give each one a grade from 1 to 5. The average of the graded tests is the starting point; a test that cannot be graded is left out of it, and the review says how many tests the score is based on.

Then I add my verdict: what the tests cannot measure, such as how the tool holds up after the test weeks, how billing and cancelling work in practice, and how it compares with the others in the category. My verdict can move the score up or down by one point at most.

Every review shows how much my verdict changed the score, and why, in its scores table.

What a grade means

5	Excellent
4	Good
3	Average
2	Below average
1	Poor

The final score is the average of the graded tests (nine when every test can be graded), to one decimal place, plus my verdict, which can move it by up to one point. If a test couldn't be finished properly it gets no grade, and the review says how many tests the score is based on. Some tools only answer the phone, or only text and chat; a channel a tool does not offer scores 1 on the channel test and is noted plainly in the review.

The 9 tests

1.Pick-up & Response Speed

All 40 calls are timed from the first ring to the greeting and checked for missed or dropped pick-ups. On 20 calls I measure the gap between the end of my sentence and the start of the reply at five points, from the recording. On the 20 web chats and 20 text conversations I time the first reply and the median reply. The phone figures set the grade; the text and chat figures are reported alongside.

5: every call answered within two rings, median phone gap under 1 second
4: every call answered, median gap 1 to 1.5 seconds
3: one or two calls missed, or median gap 1.5 to 2.5 seconds
2: three to five calls missed, or median gap 2.5 to 4 seconds
1: six or more calls missed, or median gap over 4 seconds

2.Booking & Calendar Write-back

The calendar has fixed gaps, two clinicians with different hours and two deliberate clashes. Twelve tasks book a new appointment, four move one and four cancel, spread across the three channels. After each I open the calendar or practice system and check the date, time, clinician, duration, patient name, phone number and reason. A booking in a taken slot, with the wrong clinician or outside opening hours counts as wrong. If the tool only logs a request for a person to confirm, that is reported and graded as a 3 at best.

5: 19 or 20 correct, no clashes
4: 16 to 18 correct
3: 12 to 15 correct, or bookings are requests a person has to confirm
2: 7 to 11 correct, or a double booking
1: 6 or fewer correct, or no booking at all

3.Answer Accuracy

The profile holds 20 facts: fees, hours, parking, insurance accepted, what the practice does and does not do, cancellation policy, how to reach it after hours. Each fact is asked for at least once, some twice in different words and on different channels. An answer counts as right only if it matches the profile. Invented answers, including any clinical advice, are counted separately and weigh more than a plain 'I don't know'.

5: 19 or 20 right, nothing invented
4: 17 or 18 right, at most one invented answer
3: 14 to 16 right, or two invented answers
2: 10 to 13 right, or three or four invented answers
1: under 10 right, five or more invented answers, or any clinical advice given

4.Text & Web Chat

Ten fixed tasks, from asking the opening hours to moving an appointment, are run once by text message and once in the web chat widget. Each counts as completed when the task is done without a phone call or a person stepping in. I also check whether a caller who then texts is recognised as the same person, and whether a chat can be handed to a text thread or a call.

5: all ten completed on both channels, history shared across channels
4: eight or nine completed on both
3: six or seven completed, or one channel only
2: three to five completed
1: two or fewer, or neither channel offered

5.Reminders & Confirmations

Ten appointments are set with the tool's reminder schedule. For each I record when the reminder arrived against the schedule, whether the date, time, clinician and location were right, and what happened when I replied: a confirmation, a request to reschedule, a cancellation and a question. A reminder with the wrong time, or a reply that nobody acts on, counts as a failure.

5: all ten on time and correct, every reply handled
4: nine correct, or one reply not handled
3: seven or eight correct
2: four to six correct, or a wrong time in any reminder
1: three or fewer, or no reminders

6.Check-in & Intake

Ten new patients each give the same set of details: name, date of birth, phone, email, address, insurance, reason for visit and a consent answer, two of them with awkward spellings. I check every field in the resulting record or form, whether it landed in the connected calendar or practice system without re-typing, and how long the check-in took the patient. A tool with no intake or check-in feature scores 1.

5: all ten complete and correct, in the system without re-typing
4: nine correct
3: seven or eight correct, or details arrive as an email a person must copy
2: four to six correct
1: three or fewer, or no intake feature

7.Escalation & Human Handoff

The profile names who gets emergencies, who gets complaints and what to say after hours. Ten contacts across the channels need a person: a patient in pain, a caller asking a clinical question, a complaint, a caller who refuses to talk to a machine, a chat that turns urgent. I count whether each reaches the right person, whether that person receives a summary, and what the patient hears or sees when nobody is available. Any clinical advice given instead of a handoff counts as a failure.

5: all ten reach the right person with a summary, failed transfers fall back to a message and an alert
4: nine reach the right person, or no summary is passed
3: seven or eight reach a person
2: four to six, or the patient is cut off when nobody answers
1: three or fewer, or the tool answers a clinical question itself

8.Setup & Integrations

The clock starts at sign-up and stops when a forwarded call to the practice number is answered correctly, my own fixes included. Then each point is tried, not read from a feature list: call forwarding or a new number, Google or Outlook calendar, the practice or property system the tool claims to support (read and write checked separately), a CRM, a text number, the web chat widget on a test site, after-hours rules, a second language, and an export of the week's records.

5: live within 1 hour, 9 or 10 points working
4: live within a day, 7 or 8 points
3: live within three days, 5 or 6 points
2: live within two weeks, 3 or 4 points, or setup needed a sales call and an implementation fee
1: not live within two weeks, or 2 or fewer points

9.Pricing & Value

Every plan is read from the order page, not the marketing page. The figure graded is the monthly cost of the cheapest plan that covers the tested workload: 160 calls averaging two minutes, 80 text conversations, 80 web chats and 40 reminders. Overage, per-location fees, setup fees and any charge for the practice-system connection are included. I record the billing unit, contract length, trial terms, whether a HIPAA agreement is offered at that price, and what the terms say about recordings, transcripts and whether patient data trains the vendor's models.

5: under $200 a month, monthly contract, HIPAA agreement included
4: $200 to $399 a month
3: $400 to $699, or a yearly contract, or the HIPAA agreement costs extra
2: $700 or more, or the price is only available from a sales call
1: poor value or hidden costs

Documented but not graded

Languages beyond the second one checked in the setup test, outbound recall campaigns, payment collection, insurance checks, review requests, waiting-room kiosks and the quality of the dashboard are checked and described in every review. When they work well they count in the written assessment, but they never add points to the overall grade.

Rules that apply to every test

Real calls, texts and chats to a real account and a real calendar, never promotional descriptions or demo recordings.
Identical business profiles, tasks, thresholds and sample sizes for every tool in the category.
Authorised accounts only; no password sharing, no bypassing of plan limits, no real patient data, and every test call made by me or by someone who knows it is recorded.
Every grade has evidence: a recording, a transcript, a timing, a screenshot or a table, in the full test results.
A grade changes only when a retest produces new evidence; the review records the date and the reason.

Questions about a specific grade go to the contact page. Makers can submit an AI front desk tool for testing. No AI front desk tool has been through these tests yet; every one I follow is listed on the AI front desk page, and reviews appear there as tests finish.