How I test AI image and photo editors

Ten tests, the same for every editor, from one-click web tools to the raw editors photographers use, each graded from 1 to 5 on a fixed scale. The overall grade is their average plus my verdict, which can move it by up to one point. Every test is run by hand on an account or licence I use, and the before and after files, measurements, screenshots and timings behind each grade go in the full test results published with the review.

Written by Miles Foster, Owner and tester, RankDude.

The RankDude mascot holding a laptop with a check on its screen

Each editor gets the same files, all taken by me or with the subject's permission: 12 photos of 24 megapixels (four portraits with loose hair across a range of skin tones, two pets, two glass or other see-through objects, two street scenes and two product shots), 20 mixed raw files from five camera makers (six frames of one portrait sitter and four landscapes with open sky among them) with a grey card in ten of them, 10 raw files shot at ISO 6400 to 25600 with a base-ISO tripod reference of each scene, and 50 raw files from one session for the batch test. Every editor gets the same marked objects to remove, the same ten written edit instructions and the same three timed tasks. Raw files go into editors that cannot open them as 16-bit TIFFs exported from the raw with no edits, and the review says so.

The tests are written so that a quick web editor and a desktop raw editor are judged on the same jobs: a tool built for one kind of work loses points where it cannot do the other, and the review says which kind of user each grade matters to. Desktop editors run on the same computer, whose specifications are listed in every review; every result is inspected at full size, not on the preview, and nothing is graded from a feature list or a press kit. Each review states how the account was obtained: bought at the public price, used on a free tier or trial, or provided by the vendor. A vendor-provided account never changes a grade, and a vendor never sees a review before it is published.

How I rate

Every tool gets one score out of 5.

First, I run the same 10 tests on every tool and give each one a grade from 1 to 5. The average of the graded tests is the starting point; a test that cannot be graded is left out of it, and the review says how many tests the score is based on.

Then I add my verdict: what the tests cannot measure, such as how the tool holds up after the test weeks, how billing and cancelling work in practice, and how it compares with the others in the category. My verdict can move the score up or down by one point at most.

Every review shows how much my verdict changed the score, and why, in its scores table.

What a grade means

5	Excellent
4	Good
3	Average
2	Below average
1	Poor

The final score is the average of the graded tests (ten when every test can be graded), to one decimal place, plus my verdict, which can move it by up to one point. If a test couldn't be finished properly it gets no grade, and the review says how many tests the score is based on. Some tests on a free plan or trial use fewer edits or files than usual because of limits; I test them again on a paid plan when I can.

The 10 tests

1.Auto Enhance & Colour

The automatic edit or enhance is applied to the 20 raw files and the 12 photos with no further change. On the ten raw files with a grey card in frame, the card's distance from neutral is measured in delta E (CIEDE2000) and averaged. Clipped highlights and shadows are read from the histogram, and skin tones on the four portraits are checked against the original. Each of the 32 results is then judged usable as it is, or not, against my own edit of the same file.

5: 29 to 32 usable, average delta E under 3
4: 24 to 28 usable, delta E under 5
3: 18 to 23 usable, or delta E 5 to 8
2: 10 to 17 usable, or delta E over 8
1: 9 or fewer usable, or no automatic edit

2.Object Removal & Generative Fill

Each of the 12 photos has one object marked in advance: a person, a power line, a sign, a small item on a table. I select the same area in every editor with the tool it offers and keep the first result. Three photos are then expanded by 50% of their width and three have a masked area filled from the same written instruction for every editor (for example, a bench in an empty corner). Each of the 18 results is inspected at 100% and counted as clean only with no leftover trace, smear, repeated texture, broken straight line, visible seam or perspective and lighting mismatch. Editors that only clone or heal by hand are graded on the removals and score the six fills as failed.

5: 17 or 18 of 18 clean
4: 14 to 16 clean
3: 10 to 13 clean
2: 5 to 9 clean
1: 4 or fewer clean

3.AI Masking, Cutouts & Sky Replacement

Automatic background removal on all 12 photos, with no manual touch-up; each cutout is placed on flat mid-grey and on black and checked at 200% for halos, lost hair or fur strands, chopped edges and see-through objects turned solid. On six raw files I create the editor's automatic subject, sky or person-part mask (skin, hair, clothing) and check it for missed areas and spill. The four raw landscapes also get a sky replacement, checked for edge halos and for whether the foreground light and colour were adjusted to match the new sky. Each of the 22 results either needs a manual correction or it does not; the hair and glass photos are reported separately.

5: 20 to 22 need no correction, every hair and glass photo included
4: 16 to 19
3: 11 to 15
2: 6 to 10
1: 5 or fewer, or no AI masking or cutout

4.Prompt Edit Accuracy

Ten fixed instructions on the test photos: change a jacket's colour, add a hat, turn daylight to dusk, remove text from a sign, replace the text on a sign with given words, and five more of the same kind. Each is run once and the first result counts. An edit is correct when the instruction is carried out and the rest of the image is unchanged when compared side by side at full size. An editor with no written instructions may do each edit with its own AI tools, and any edit it cannot do counts as failed.

5: 9 or 10 of 10 correct
4: 7 or 8 correct
3: 5 or 6 correct
2: 3 or 4 correct
1: 2 or fewer correct

5.Portrait Retouching

On the four portraits I run blemish removal, skin smoothing at the default strength, eye and teeth brightening, and relighting. Each of the 16 results is compared with the original at 100%: the person must still look like themselves side by side, skin tone must not shift, pores and fine skin texture must remain, and nothing may look painted or plastic. One retouch is then made on the first raw frame of the portrait sitter, copied to the other five and checked for consistency from frame to frame. Overdone and failed edits are counted.

5: all 16 natural, likeness and texture kept, sync consistent
4: one or two overdone
3: three to five overdone, or no way to sync a retouch
2: likeness or skin tone changed on any portrait
1: no face tools or unusable results

6.Denoise, Sharpen & Upscale

The ten high-ISO raw files are denoised at the default setting, with the editor's sharpening or deblur on where it offers one, and compared at 100% with the base-ISO tripod shot of the same scene. Six of the test photos are downsized to 512 pixels on the long edge and upscaled 4x at the highest setting the plan allows, then compared with the full-size original. Each of the 16 results is checked for fabric, hair and small text detail, waxy skin, worm-like artefacts, halos, colour shifts, changed faces and invented texture, and counts as clean only when none of these shows. SSIM against the reference is measured and reported alongside the grade.

5: 15 or 16 of 16 clean
4: 12 to 14 clean
3: 8 to 11 clean
2: 4 to 7 clean, or only one of denoise and upscale offered
1: 3 or fewer clean, or neither offered

7.Speed per Edit & Batch

Every edit in the removal, masking and prompt tests is timed from pressing apply or generate until the finished result is shown at full quality; progress bars and low-resolution previews do not count. That is at least 50 edits on the same computer and connection. Then the 50 raw files are imported, given the automatic edit plus one automatic subject mask, and exported as full-size JPEGs, with the clock running from starting the import to the last file written on disk, upload and download included for cloud editors. An editor with no batch does the files one by one. Each figure gets its own grade on the scale below and the lower of the two counts; the average, fastest, slowest, crashes and skipped files are reported.

5: median under 5 seconds, batch under 3 minutes
4: 5 to 9.9 seconds, batch under 6 minutes
3: 10 to 19.9 seconds, batch under 12 minutes
2: 20 to 44.9 seconds, batch under 25 minutes
1: 45 seconds or longer, batch 25 minutes or longer, or the batch failed

8.File Support & Export

Each point is tried with real files on the free tier and on a paid plan, not read from the feature list: raw files from all five camera makers open, raw files from a camera released in the last 12 months open, HEIC, TIFF and layered files open, export at the full input resolution, no watermark or visible credit on paid exports, no watermark on free exports, a lossless export (PNG or TIFF), 16-bit TIFF or DNG export, metadata kept on export, and no daily or monthly cap on paid downloads.

5: all ten
4: 8 or 9
3: 6 or 7
2: 4 or 5, or exports under 2,048 pixels on the long edge
1: 3 or fewer, or no clean download

9.Ease of Use & Control

From opening the editor with a new photo, I remove an object, swap the background for plain white, and apply the automatic edit to a raw file and export it. Each click, menu, dialog and upload counts as a step; the time to a downloaded file is recorded, and any help page or dead end is noted. Then each AI tool is checked on a real file for eight controls: adjustable strength, a mask or selection I can refine by hand, sliders still usable after the automatic edit, full undo history, the original file left untouched, a before and after view, settings copied to other files, and saved presets.

5: every task under 5 steps and 1 minute, all eight controls
4: under 8 steps each, six or seven controls
3: under 12 steps, or help needed once, or four or five controls
2: 12 or more steps on a task, or three or fewer controls
1: a task could not be finished

10.Pricing & Value

Every plan, licence and credit pack is read from the checkout, not the marketing page. Each is turned into a three-year cost in dollars for a fixed workload of 200 AI edits (removals, cutouts, fills and upscales) and 500 raw or full-size photos a month, on the cheapest plan that covers it; one-off licences include the upgrade fees announced in that time. The cost of one edit on that plan is reported too. I also check what still works if the subscription ends: whether edits stay readable and files stay exportable. Free-tier limits, renewal terms and payment methods are listed as seen at checkout.

5: under $250 over three years, nothing locked when you stop paying
4: $250 to $399, or one weak spot
3: $400 to $599
2: $600 or more, or edits locked when you stop paying
1: poor value or hidden costs

Documented but not graded

AI culling, learned personal editing styles, layers, text-to-image generation from scratch, mobile apps, cloud sync between devices, plugins for or round trips with other editors, and an API are checked and described in every review. When they work well they count in the written assessment, but they never add points to the overall grade.

Rules that apply to every test

Real use of the product on a real account or licence, never promotional descriptions or sample galleries.
Identical photos, raw files, instructions, thresholds, sample sizes and computer for every editor in the category.
Authorised accounts and licences only; no password sharing, no shared keys or cracked builds, no bypassing of usage limits, trial limits or bot checks, and only photos I own or have permission to edit.
Every grade has evidence: the original and edited files, a measurement, a screenshot or a table, in the full test results.
A grade changes only when a retest produces new evidence; the review records the date and the reason.

Questions about a specific grade go to the contact page. Makers can submit an AI image editor for testing. No AI image editor has been through these tests yet; every one I follow is listed on the AI image editors page, and reviews appear there as tests finish.