RankDude

How I test AI scheduling assistants

Nine tests, the same for every assistant, each graded from 1 to 5 on a fixed scale. The overall grade is their average plus my verdict, which can move it by up to one point. Every test is run by hand on an account I use, with real meetings booked by real people in real time zones, and the calendar screenshots, email threads, timings and tables behind each grade go in the full test results published with the review.

Written by Miles Foster, Owner and tester, RankDude.

Each assistant gets the same two calendars to manage: a Google calendar and an Outlook calendar for the same imaginary consultant, already holding a fixed week of meetings, two deliberate clashes, working hours that differ by day, and a list of 15 tasks with deadlines. Then it gets a fortnight of scheduling work: 12 bookings through its public page from five time zones, ten meeting requests by email or chat involving other people, ten changes and cancellations, ten reminders to send, and a week of tasks to fit in. The other participants are five colleagues, one in each of the five time zones, who book, reply and change their minds on a script. Nothing is graded from a feature list or a demo video.

Each review states how the account was obtained: bought at the public price, used on a free tier, or provided by the vendor. A vendor-provided account never changes a grade, and a vendor never sees a review before it is published.

How I rate

Every tool gets one score out of 5.

First, I run the same 9 tests on every tool and give each one a grade from 1 to 5. The average of the graded tests is the starting point; a test that cannot be graded is left out of it, and the review says how many tests the score is based on.

Then I add my verdict: what the tests cannot measure, such as how the tool holds up after the test weeks, how billing and cancelling work in practice, and how it compares with the others in the category. My verdict can move the score up or down by one point at most.

Every review shows how much my verdict changed the score, and why, in its scores table.

What a grade means

5Excellent
4Good
3Average
2Below average
1Poor

The final score is the average of the graded tests (nine when every test can be graded), to one decimal place, plus my verdict, which can move it by up to one point. If a test couldn't be finished properly it gets no grade, and the review says how many tests the score is based on. An assistant built only for booking appointments, with no task planning, gets no grade on the task test rather than a 1, and the review says so; an assistant with no public booking page is graded 1 on that test, because finding a time for someone else is the job.

The 9 tests

1.Booking Page

The five colleagues, one in each time zone, make twelve bookings through the public page, including two attempts on a slot that is already taken, two on a slot inside the buffer after another meeting, and one outside working hours. After each I check the date, time, duration, invitee details and video link on both calendars, and that the invitee's confirmation shows the time in their own zone. A booking in a taken slot, inside a buffer or outside hours counts as wrong.

  • 5: all twelve correct, every blocked attempt refused
  • 4: ten or eleven correct
  • 3: eight or nine correct, or a buffer ignored
  • 2: five to seven correct, or a double booking
  • 1: four or fewer correct, or no public booking page

2.Scheduling by Email or Chat

Ten requests are made the way a person would make them: an email copying the assistant to a colleague asking for 45 minutes next week, a chat message asking it to find a time with two people, a reply saying mornings are no good, a request for a time that does not exist. For each I count the rounds of messages before an invite arrives, time the gap from the first request to the invite on the calendar, and check the invite against what was asked. An assistant that only offers a link counts as a partial.

  • 5: all ten booked correctly within two rounds, median under 30 minutes
  • 4: eight or nine booked, or median under 2 hours
  • 3: six or seven booked, or the assistant only sends a link
  • 2: three to five booked, or a wrong time in any invite
  • 1: two or fewer, or no email or chat scheduling

3.Time Zones & Working Hours

The ten cases include a three-person meeting across London, New York and Sydney, a booking in the week the clocks change in one zone but not another, a participant whose working hours end at three on Fridays, a travel day with a changed zone, and an invitee who gives their zone wrongly. For each I check the time shown to every participant and whether it falls inside the hours each one set. A slot shown in the wrong zone to anyone, or outside anyone's hours, counts as wrong.

  • 5: all ten right
  • 4: nine right
  • 3: seven or eight right
  • 2: five or six right, or a clock-change error
  • 1: four or fewer, or working hours cannot be set per day

4.Rescheduling & Cancellations

Five changes come from invitees: a reschedule through the link, a cancellation, a reply asking to move, a no-show that needs a new time, a request to add a person. Five come from me: moving a meeting into a clash, cancelling a recurring meeting once, shortening a meeting, changing the video link, and a priority meeting that must displace another. For each I check both calendars, every participant's invite and the assistant's own record, and note anything left behind.

  • 5: all ten handled, every calendar and person updated
  • 4: eight or nine handled
  • 3: six or seven handled, or a participant not told
  • 2: three to five handled, or a slot left blocked after a cancellation
  • 1: two or fewer, or changes must be made by hand in both calendars

5.Reminders & Confirmations

Ten meetings are set with the assistant's reminder schedule by email and, where offered, text. For each I record when the reminder arrived against the schedule, whether the time, link and location were right and shown in the recipient's zone, and what happened when the recipient replied: a confirmation, a request to move, a cancellation and a question. A reminder with the wrong time, or a reply that nobody acts on, counts as a failure.

  • 5: all ten on time and correct, every reply handled
  • 4: nine correct, or one reply not handled
  • 3: seven or eight correct
  • 2: four to six correct, or a wrong time in any reminder
  • 1: three or fewer, or no reminders

6.Calendar Sync & Conflicts

Ten events are created or moved directly in Google Calendar and Outlook and I time how long each takes to appear in the assistant and in the other calendar. Ten are created in the assistant and timed the other way. The two planted clashes are checked: the assistant must see them as busy and refuse to book over them. Duplicate events, ghost holds left after a cancellation and events that appear with the wrong duration are counted as faults.

  • 5: all twenty within 2 minutes, both clashes caught, no faults
  • 4: all twenty within 15 minutes, or one fault
  • 3: within an hour, or two or three faults
  • 2: longer than an hour, or a clash booked over, or duplicates
  • 1: one calendar only, or sync has to be refreshed by hand

7.Task & Focus Time Planning

The fifteen tasks have deadlines, durations and three priorities, and two depend on another finishing first. I check where each is placed, that none sits outside working hours or over a meeting, and that the dependent tasks come after their parents. Then a two-hour meeting is dropped into the middle of the busiest day and I check what the assistant moves, what it leaves late, and whether it tells me. An assistant with no task planning gets no grade on this test.

  • 5: all fifteen placed correctly, re-planned after the disruption with nothing silently late
  • 4: thirteen or fourteen correct, or one task left late without a warning
  • 3: ten to twelve correct
  • 2: six to nine correct, or tasks placed over meetings
  • 1: five or fewer correct

8.Setup & Integrations

The clock starts at sign-up and stops when a booking made through the assistant appears on both calendars with a working video link, my own fixes included. Then each point is tried, not read from a feature list: Google Calendar, Outlook, a second calendar on the same account, Zoom, Google Meet or Teams links created automatically, a CRM, Zapier or a webhook, payment at booking, a booking widget embedded on a test site, round-robin or collective booking for a team, and an export of all bookings.

  • 5: live within 20 minutes, 9 or 10 points working
  • 4: live within an hour, 7 or 8 points
  • 3: live the same day, 5 or 6 points
  • 2: live within a week, 3 or 4 points, or setup needed a sales call
  • 1: not live within a week, or 2 or fewer points

9.Pricing & Value

Every plan is read from the checkout, not the marketing page. The figure graded is the yearly cost of the cheapest plan that connects two calendars, gives a public booking page with the assistant's branding removed, sends reminders and includes the AI scheduling feature the product is sold on. The per-seat price for five people, any charge for text reminders, the trial, whether a card is needed, the renewal price against the first-year price and what the terms say about calendar and email data are recorded alongside.

  • 5: under $120 a year, same price at renewal
  • 4: $120 to $199 a year
  • 3: $200 to $299, or the AI feature costs extra
  • 2: $300 or more, or the price is only available from a sales call
  • 1: poor value or hidden costs

Documented but not graded

Meeting notes and recordings, routing forms, languages beyond English, mobile apps, analytics and team features such as shared availability are checked and described in every review. When they work well they count in the written assessment, but they never add points to the overall grade.

Rules that apply to every test

  • Real bookings on real calendars with real participants, never promotional descriptions or demo videos.
  • Identical calendars, requests, thresholds and sample sizes for every assistant in the category.
  • Authorised accounts only; no password sharing, no bypassing of plan limits, and test payments only in test mode or refunded.
  • Every grade has evidence: a screenshot, an email thread, a timing or a table, in the full test results.
  • A grade changes only when a retest produces new evidence; the review records the date and the reason.

Questions about a specific grade go to the contact page. Makers can submit an AI scheduling assistant for testing. No AI scheduling assistant has been through these tests yet; every one I follow is listed on the AI scheduling assistants page, and reviews appear there as tests finish.