How I test AI marketing automation

Nine tests, the same for every platform, each graded from 1 to 5 on a fixed scale. The overall grade is their average plus my verdict, which can move it by up to one point. Every test is run by hand on an account I use, the campaigns are sent for real to a test list, and the timings, seed-inbox results, screenshots and checklists behind each grade go in the full test results published with the review.

Written by Miles Foster, Owner and tester, RankDude.

The RankDude mascot holding a laptop with a check on its screen

Each platform gets the same business: a small online shop with a test list of 2,000 contacts, two years of invented but consistent order history, a connected test store and a website. It runs the same three campaigns: a product launch, a win-back for customers who have not bought in six months, and a welcome journey for new sign-ups. Each campaign brief holds the facts the copy must get right. I build the segments, write with the platform's AI, send to seed inboxes, run the journeys with test contacts and read the reports. Nothing is graded from a feature list or an analyst chart.

Each review states how the account was obtained: bought at the public price, used on a free tier, or provided by the vendor. A vendor-provided account never changes a grade, and a vendor never sees a review before it is published.

How I rate

Every tool gets one score out of 5.

First, I run the same 9 tests on every tool and give each one a grade from 1 to 5. The average of the graded tests is the starting point; a test that cannot be graded is left out of it, and the review says how many tests the score is based on.

Then I add my verdict: what the tests cannot measure, such as how the tool holds up after the test weeks, how billing and cancelling work in practice, and how it compares with the others in the category. My verdict can move the score up or down by one point at most.

Every review shows how much my verdict changed the score, and why, in its scores table.

What a grade means

5	Excellent
4	Good
3	Average
2	Below average
1	Poor

The final score is the average of the graded tests (nine when every test can be graded), to one decimal place, plus my verdict, which can move it by up to one point. If a test couldn't be finished properly it gets no grade, and the review says how many tests the score is based on. Some tests on a free plan can't send SMS or connect a store; I test them again on a paid plan when I can.

The 9 tests

1.Setup to First Send

The clock starts at account creation and stops when the launch campaign has been sent to all 2,000 contacts, with the test store and the website tracking connected and the sending domain authenticated. Import, domain setup, store connection and any verification wait are all on the clock. I note every step that needed support or a developer.

5: under 2 hours, no help
4: 2 to 4 hours
3: 4 to 8 hours, or one step needed support
2: 1 to 3 days
1: over 3 days, or the store could not be connected

2.AI Copy Quality

For each campaign the AI writes an email, two social posts and a landing page from the brief: twelve pieces. Each brief holds ten facts (prices, dates, product names, the offer, the deadline). I check every fact in the output, count invented claims (reviews, figures, guarantees not in the brief), and count faults: wrong brand name, broken merge fields, spelling errors, text over the length limit and subject lines over 60 characters.

5: 30 facts right, nothing invented, 0 to 2 faults
4: 27 to 29 facts right, at most one invented item, 3 to 5 faults
3: 22 to 26 facts right, or two or three invented items
2: 15 to 21 facts right, or invented reviews or guarantees
1: under 15 facts right, or no AI writing

3.Segmentation & Triggers

The test list is built so I know the right count for every segment. I build ten: bought in the last 30 days, no order in six months, spent over $200 in total, abandoned a cart in the last week, opened the last three emails but never bought, lives in one country, bought one product but not its accessory, signed up this month, clicked a given link, and a predicted-to-churn or lookalike segment where offered. A segment counts when it can be built without support and its count matches mine within 1%.

5: all ten correct
4: 8 or 9
3: 6 or 7
2: 3 to 5
1: 2 or fewer

4.Journey Automation

The journey is the same for every platform: welcome email on sign-up, wait two days, branch on whether the first email was opened, a second email or an SMS depending on the branch, a wait until a product is viewed, a discount email, exit when an order is placed, and a tag written back to the contact. I enrol 20 test contacts who behave in known ways and check every contact's path and timing against the plan.

5: all eight steps and all 20 paths right
4: one step missing or one path wrong
3: two steps missing, or up to four paths wrong
2: three or more steps missing
1: the journey could not be built

5.Channel Coverage

Each point is tried for real: an email campaign, an SMS, a web push or app push, a social post published to a connected account, an audience synced to Meta or Google Ads, a landing page published, a sign-up form embedded on the test site, a WhatsApp or Messenger message, an on-site pop-up or banner, and an A/B test with a declared winner. Each counts when it works end to end on the plan tested.

5: all ten
4: 8 or 9
3: 6 or 7
2: 3 to 5
1: 2 or fewer

6.Deliverability & Compliance

The seed list has 20 inboxes across Gmail, Outlook, Yahoo and iCloud. For each of the three campaigns I record where the email landed: inbox, promotions, spam or missing. I check that SPF, DKIM and DMARC pass in the headers, that the unsubscribe link and the one-click header both work and take effect before the next send, that a double opt-in form is available, and that a contact's data can be exported and deleted on request.

5: 90% or more in the inbox, all checks pass
4: 80 to 89%
3: 65 to 79%, or one compliance check fails
2: 50 to 64%, or an unsubscribe not honoured
1: under 50%, or authentication cannot be set up

7.Reporting & Attribution

The test store records every order, so I know the right answers. The ten questions: revenue from each campaign, open and click rate per campaign, the segment with the highest conversion, revenue from the welcome journey, unsubscribes per send, the best-performing subject line in the A/B test, orders attributed to SMS, revenue by channel for the month, the AI's stated reason for any recommendation, and a CSV export of the campaign results. An answer counts when the figure matches the store within 5%.

5: all ten right
4: 8 or 9
3: 6 or 7
2: 3 to 5
1: 2 or fewer

8.Integrations & CRM Sync

Each point is set up and checked: the test store's orders and products syncing in, a two-way contact sync with HubSpot or Salesforce, a webhook or Zapier trigger firing on a sign-up, Google Analytics 4 events for a campaign, a Meta or Google Ads audience that updates, an API call that creates a contact, a full export of contacts and events to CSV, and an imported list with custom fields mapped correctly.

5: all eight
4: 6 or 7
3: 4 or 5
2: 2 or 3
1: 1 or none

9.Pricing & Value

Every plan is read from the checkout, not the marketing page. The figure graded is the yearly cost of the cheapest plan that includes the AI writing and journey features, the store connection and 2,000 contacts. The cost at 10,000 contacts, any send limits, SMS and AI credit charges, seat fees, the renewal price against the first-year price and the payment methods seen at checkout are listed.

5: under $600 a year at 2,000 contacts, AI included, same price at renewal
4: $600 to $1,199
3: $1,200 to $2,399, or AI features cost extra
2: $2,400 or more, or quote-only pricing
1: poor value or hidden costs

Documented but not graded

Templates, the email editor, image generation, predictive send times, a customer data platform, multilingual campaigns and support are checked and described in every review. When they work well they count in the written assessment, but they never add points to the overall grade.

Rules that apply to every test

Real use of the product on a real account, real sends and real test data, never promotional descriptions or analyst charts.
Identical test lists, briefs, tests, thresholds and sample sizes for every platform in the category.
Authorised accounts only; no password sharing, no bypassing of plan limits or bot checks, and sends only to seed inboxes and test contacts I control.
Every grade has evidence: a measurement, a seed-inbox result, a screenshot or a checklist, in the full test results.
A grade changes only when a retest produces new evidence; the review records the date and the reason.

Questions about a specific grade go to the contact page. Makers can submit an AI marketing automation tool for testing. No AI marketing automation tool has been through these tests yet; every one I follow is listed on the AI marketing automation page, and reviews appear there as tests finish.