How I test AI video generators

Ten tests, the same for every generator, each graded from 1 to 5 on a fixed scale. The overall grade is their average plus my verdict, which can move it by up to one point. Every test is run by hand on an account I use, from the same prompts, images and scripts, and the clips, scoring sheets, timings and cost sheets behind each grade go in the full test results published with the review.

Written by Miles Foster, Owner and tester, RankDude.

The RankDude mascot holding a laptop with a check on its screen

Each generator gets the same fixed set, written before any tool is opened: 20 text prompts (people talking and walking, hands at work, animals, a car chase, liquids and fire, a product turning on a table, a crowded street, text on a sign, a two-shot camera move), 10 still images to animate (a portrait, a product photo, a painting, a landscape, a group photo I have the rights to), and 3 scripts of about 60 seconds (a product explainer, a training lesson, a social short). Clip generators get the prompts and images; avatar and script-to-video tools get the scripts and, where they also take prompts, the prompts too. Every clip is made at the best setting the plan under test allows, saved with its prompt, settings, credits used and time, and graded at full size on a calibrated screen. Nothing is graded from a showcase reel.

Each review states how the account was obtained: bought at the public price, used on a free tier, or provided by the vendor. A vendor-provided account never changes a grade, and a vendor never sees a review before it is published.

How I rate

Every tool gets one score out of 5.

First, I run the same 10 tests on every tool and give each one a grade from 1 to 5. The average of the graded tests is the starting point; a test that cannot be graded is left out of it, and the review says how many tests the score is based on.

Then I add my verdict: what the tests cannot measure, such as how the tool holds up after the test weeks, how billing and cancelling work in practice, and how it compares with the others in the category. My verdict can move the score up or down by one point at most.

Every review shows how much my verdict changed the score, and why, in its scores table.

What a grade means

5	Excellent
4	Good
3	Average
2	Below average
1	Poor

The final score is the average of the graded tests (ten when every test can be graded), to one decimal place, plus my verdict, which can move it by up to one point. A test that does not apply to a tool (lip sync on a generator that makes no speech) or that could not be finished properly gets no grade, and the review says how many tests the score is based on. Free plans often cap length, resolution or watermark the output; I test again on a paid plan when I can.

The 10 tests

1.Prompt Accuracy

Each of the 20 prompts is split in advance into its checkable elements (about 100 in all: who, doing what, where, in which light, with which camera move). Each prompt is generated once, first attempt only, and I tick the elements that appear correctly for most of the clip. The grade is the share ticked across all prompts; the number of outright refusals and blocked prompts is reported alongside.

5: 90% or more of elements correct
4: 75% to 89%
3: 60% to 74%
2: 40% to 59%
1: under 40%, or most prompts refused

2.Motion & Realism

Every clip from the prompt and image tests is watched at full size, frame by frame where needed, and marked for faults from a fixed list: extra or merging fingers and limbs, faces that melt or change, objects passing through each other, liquids or fire behaving wrongly, sliding feet, things that pop in or vanish, warped text and backgrounds. The grade is the share of clips with no fault; the most common faults are listed.

5: 80% or more of clips clean
4: 60% to 79%
3: 40% to 59%
2: 20% to 39%
1: under 20%

3.Image to Video

Each of the 10 images is animated with the same short motion prompt. I compare the last frame with the source: the person must still be the same person, the product's logo, shape and colour must not drift, and the framing must hold unless a camera move was asked for. Tools that cannot start from an image get no grade here.

5: 9 or 10 of 10 faithful
4: 7 or 8
3: 5 or 6
2: 3 or 4
1: 2 or fewer

4.Consistency Across Shots

I make a five-shot scene twice (a woman ordering coffee; a product being unboxed and used), using whatever the tool offers to keep things consistent: reference images, saved characters, scene or storyboard tools, or the same seed. Ten viewers who have not seen the prompts are shown the shots and asked whether each is the same person and the same product; the grade comes from their answers.

5: same in all five shots for nearly every viewer
4: one shot drifts
3: two shots drift
2: only the outfit or setting holds
1: no way to keep a character across shots

5.Voice, Sound & Lip Sync

I generate the three scripts as speech (with the tool's avatar, a generated character or a lip-synced image), plus five clips that need sound effects. I check lip sync frame by frame on ten sentences, rate voice naturalness on a fixed five-point sheet in English, Spanish and one more language, and note whether sound effects and music fit the action. Tools with no audio at all get no grade here, and the review says so.

5: in sync within two frames, natural in three languages, fitting sound
4: small sync slips or one weak language
3: audibly robotic voice or drifting sync
2: audio only added afterwards from a stock library
1: speech does not match the mouth

6.Editing & Camera Control

On one clip I try the same ten changes: set a camera move, set the first and last frame, extend the clip by five seconds, change one object, change the background, slow a moment down, change the aspect ratio, swap the voice or a line of the script, trim and reorder shots, and upscale. Each counts if it works inside the tool on the plan under test and does what was asked.

5: 8 to 10 changes work
4: 6 or 7
3: 4 or 5
2: 2 or 3
1: one or none; the only fix is a new prompt

7.Length, Resolution & Export

I generate at the maximum length and resolution offered, check the real resolution, frame rate and bitrate of the downloaded file (not what the menu says), whether 4K is a true render or an upscale, whether a watermark or visible label remains, and whether the export carries audio, captions and a vertical version where offered.

5: 10 s or more, true 1080p or above, clean export
4: 5 to 9 s, or 1080p only by upscale
3: 720p, or a visible label on paid output
2: under 5 s, or below 720p
1: watermark on paid plans or no download

8.Speed & Reliability

Every generation in the other tests is timed from submit to a downloadable file, at three times of day on weekdays. I report the median and the slowest tenth, count failed or stuck generations and unexplained refusals, and note whether failed attempts still used credits.

5: median under 2 min, 1 failure in 50 or fewer
4: 2 to 5 min
3: 5 to 10 min, or several failures
2: over 10 min, or one in ten failing
1: generations regularly fail or never finish

9.Usage Rights & Safety

I read the terms, the acceptable-use policy and the plan pages and record who owns the output, whether commercial use is allowed on each plan, whether the tool trains on my uploads by default and how to opt out, what it requires before making an avatar or voice of a real person, whether it attaches C2PA or another provenance label, and whether it refuses a fixed set of five prompts that ask for a real public figure or a deceptive news clip.

5: commercial use clear, consent checks, provenance labels, deceptive prompts refused
4: one of those weaker or unclear
3: commercial use only on top plans, or trains on uploads with no opt-out
2: unclear ownership, or a real person's likeness made without any check
1: no commercial use on any plan, or terms that claim the user's uploads

10.Pricing & Cost per Second

Every plan is read from the checkout, not the marketing page. I log the credits each generation used, including failed and discarded attempts, and divide the month's spend by the seconds of video that passed the motion test. I note whether the cost is shown before generating, what happens when credits run out, whether unused credits roll over, the renewal price against any first-month price, and the payment methods seen at checkout.

5: under $0.10 per usable second, cost shown beforehand, failures not charged
4: $0.10 to $0.29
3: $0.30 to $0.59, or cost only shown afterwards
2: $0.60 to $0.99, or failed generations charged
1: $1 or more per usable second, or no way to see what a clip cost

Documented but not graded

Templates, stock libraries, brand kits, team workspaces, an API, plugins for editors, avatars made from your own footage, dubbing and translation, mobile apps and the range of styles are checked and described in every review. When they work well they count in the written assessment, but they never add points to the overall grade.

Rules that apply to every test

Real use of the product on a real account, never showcase reels or a vendor's demo clips.
Identical prompts, images, scripts, fault lists, thresholds and sample sizes for every generator in the category.
Authorised accounts only; no password sharing, no bypassing of plan limits, content filters or bot checks; no likeness of a real person is used without their written consent.
Every grade has evidence: the clips themselves, a scoring sheet, a timing log or a cost sheet, in the full test results.
A grade changes only when a retest produces new evidence; the review records the date and the reason.

Questions about a specific grade go to the contact page. Makers can submit an AI video generator for testing. No AI video generator has been through these tests yet; every one I follow is listed on the AI video generators page, and reviews appear there as tests finish.