How I test AI marketing agents

Ten tests, the same for every agent, each graded from 1 to 5 on a fixed scale. The overall grade is their average plus my verdict, which can move it by up to one point. Every test is run by hand on an account I use, against real social, advertising and analytics accounts that belong to a test business, and the timings, counts, screenshots and transcripts behind each grade go in the full test results published with the review.

Written by Miles Foster, Owner and tester, RankDude.

The RankDude mascot holding a laptop with a check on its screen

Each agent gets the same month of marketing work for the same test business: a small online shop with a live website, Instagram, Facebook, LinkedIn, X, TikTok and YouTube accounts, a Google Ads account and a Meta Ads account with a small fixed budget, a GA4 property and an email list. The brief holds 20 facts about the business, a one-page brand voice sheet and a list of ten banned words. I connect the agent to those accounts, hand it the brief, and run the tasks below on what it actually publishes, spends and reports. Nothing is graded from a feature list, a demo video or a sales call. The category is for agents that take on this work across channels; social media schedulers are tested under AI content automation and SEO tools under AI SEO automation, and they are only listed here.

Each review states how the account was obtained: bought at the public price, used on a free tier, or provided by the vendor. A vendor-provided account never changes a grade, and a vendor never sees a review before it is published.

How I rate

Every tool gets one score out of 5.

First, I run the same 10 tests on every tool and give each one a grade from 1 to 5. The average of the graded tests is the starting point; a test that cannot be graded is left out of it, and the review says how many tests the score is based on.

Then I add my verdict: what the tests cannot measure, such as how the tool holds up after the test weeks, how billing and cancelling work in practice, and how it compares with the others in the category. My verdict can move the score up or down by one point at most.

Every review shows how much my verdict changed the score, and why, in its scores table.

What a grade means

5	Excellent
4	Good
3	Average
2	Below average
1	Poor

The final score is the average of the graded tests (ten when every test can be graded), to one decimal place, plus my verdict, which can move it by up to one point. If a test couldn't be finished properly it gets no grade, and the review says how many tests the score is based on. An agent that does not touch advertising or SEO is still graded on those tests, because the category promises them; the review says so plainly.

The 10 tests

1.Setup to First Task

The clock starts when I create the account and stops when the agent has published, scheduled or sent its first real item on one of the test business's connected channels. I note whether a developer, a sales call or an onboarding session was required, and how many of the brief's documents the agent could take in at setup.

5: under 30 minutes, self-serve
4: 30 to 59 minutes, self-serve
3: 1 to 4 hours, or an onboarding call needed
2: more than 4 hours, or a developer needed
1: more than a day, or no self-serve access at all

2.Connected Channels

Each connection counts only when the agent can read from and act on the account, not when it appears in an integrations list: Instagram, Facebook, LinkedIn, X, TikTok, YouTube, Google Ads, Meta Ads, GA4, Google Search Console, the email platform and the website CMS. A connection that reads but cannot post or change anything counts as half, and a half is rounded down in the total.

5: all twelve
4: 10 or 11
3: 7 to 9
2: 4 to 6
1: 3 or fewer

3.Does It Finish the Job

The same ten tasks for every agent: publish a post, schedule a full week of posts across three channels, launch a paused ad campaign in Google Ads, launch a paused ad campaign in Meta Ads, send an email campaign to the test list, reply to five comments, pause the worst-performing ad, publish a blog post to the site, rewrite ten meta descriptions, and add UTM tags to every link in a campaign. A task counts when the result is live and correct; a draft I have to copy and paste does not count.

5: all ten
4: 8 or 9
3: 6 or 7
2: 3 to 5
1: 2 or fewer

4.Content Quality & Accuracy

From the brief the agent produces ten social posts, five ad sets, five marketing emails, five blog introductions and five video captions. Each piece is checked for faults: a fact that contradicts the brief, an invented claim, price, review or statistic, a banned word, text that could belong to any business, placeholder text, broken formatting, and tone that breaks the voice sheet. Every fault is listed in the test results with the piece it was found in.

5: 0 to 3 faults, nothing invented
4: 4 to 7 faults, at most one invented item
3: 8 to 14 faults, or two or three invented items
2: 15 to 25 faults, or invented reviews or statistics
1: 26 or more faults

5.Ad Management

The agent gets a fixed budget and the same goal on both ad accounts for 14 days. Eight tasks are checked: build the campaign, set the daily budget, write the ads from the brief, choose audiences or keywords, add conversion tracking, report results daily, change the budget when I ask, and pause an ad when I ask. Spend is read from the ad platforms, not from the agent; any spend above the cap is recorded, and a breach caps the grade as the scale below sets out.

5: all eight tasks, spend within the cap
4: 6 or 7 tasks, spend within the cap
3: 4 or 5 tasks, or spend up to 10% over the cap
2: 2 or 3 tasks, or spend more than 10% over the cap
1: no campaign launched, or spend more than 25% over the cap

6.SEO Tasks

The same eight tasks for every agent: keyword research from one seed term, a content brief for it, a draft article that follows the brief, new titles and meta descriptions for ten pages, internal link suggestions, a fix for one indexing problem shown in Search Console, a technical audit summary of the site, and a rank report for ten tracked terms. Each task counts when the output is correct when checked, not when it is merely delivered; a rank report is compared with my own tracker.

5: all eight
4: 6 or 7
3: 4 or 5
2: 2 or 3
1: 1 or none

7.Reporting Accuracy

Each week the agent is asked for the same report: followers gained, reach, engagements, link clicks, ad spend, ad clicks, conversions, email open rate, site sessions and organic clicks. Each figure is compared with the platform it comes from on the same day. A figure counts when it is within 2% of the source. I also record whether the report arrived on schedule and whether the agent explained any gap it could not fill.

5: all ten within 2%, every report on time
4: 8 or 9 within 2%
3: 6 or 7 within 2%, or a report late
2: 3 to 5 within 2%, or a figure invented
1: 2 or fewer, or no report at all

8.Approvals & Safety Controls

Each control is tested, not read from the settings page: approval before publishing, approval before any spend, a hard spend cap, access limited to chosen channels, access that can be revoked from the connected account, a log of every action with a time, undo or delete of a post the agent made, a pause button that stops all activity, a banned-topics list that is honoured, and a route to a human when the agent is unsure. A control counts when I try to break it and it holds.

5: all ten
4: 8 or 9
3: 6 or 7
2: 3 to 5
1: 2 or fewer, or the agent published or spent without asking

9.Learning From Corrections

In the first week I give the agent ten corrections through its own interface: three on tone, three on facts, two on banned words, one on posting time and one on hashtag use. Every piece of content in weeks two to four is then checked for a repeat of each corrected mistake. The count of repeats is the grade; how the agent stores and shows what it has learned is described in the review.

5: no repeats
4: 1 or 2 repeats
3: 3 to 5 repeats
2: 6 to 9 repeats
1: 10 or more repeats, or corrections cannot be given

10.Pricing & Value

Every plan is read from the checkout, not the marketing page. I record the cheapest plan that can run the full month of work, the credits or actions it consumed, any overage charged, per-seat and per-channel fees, and the renewal price against the first-year price. The figure graded is the yearly cost of running the test month twelve times. A product with no published price and no self-serve plan is graded on what sales quoted me, and the review says so.

5: under $600 a year, published prices, no overage
4: $600 to $1,199 a year, or under $600 with the price only from sales
3: $1,200 to $2,999 a year, or renewal more than 50% higher
2: $3,000 to $5,999 a year
1: $6,000 or more a year, or hidden costs

Documented but not graded

Image and video generation, influencer tools, listening and sentiment, multilingual output, team seats and the quality of the agent's own suggestions are checked and described in every review. When they work well they count in the written assessment, but they never add points to the overall grade.

Rules that apply to every test

Real use of the product on a real account and real connected channels, never promotional descriptions, demo videos or case studies.
Identical brief, tasks, budget, thresholds and sample sizes for every agent in the category.
Authorised accounts only; no password sharing, no bypassing of plan limits or platform rules, and ad spend only from my own capped test budget.
Every grade has evidence: a timing, a count, a screenshot, a log or a transcript, in the full test results.
A grade changes only when a retest produces new evidence; the review records the date and the reason.

Questions about a specific grade go to the contact page. Makers can submit an AI marketing agent for testing. No AI marketing agent has been through these tests yet; every one I follow is listed on the AI marketing agents page, and reviews appear there as tests finish.