Checker · AI app builder

Spec check

Every AI app builder gets the same brief, a pilates studio's booking app, and the same list of thirty checks written before the test. The spec check runs those checks against the app the builder published and counts what passes, so the accuracy grade is a count, not an impression.

When it runs: After the first build and after every change
Used in: 20 reviews
Runs on: My PC against the published app, with my own results where a check needs judgement

The brief fixes ten facts about the studio, thirty checks the finished app must pass, twelve changes to ask for afterwards, and two attacks. The checks range from the timetable showing the right days and times to a cancelled booking giving a credit back. Six of the thirty need the reader's own payment or email keys and are left out, so the figure is out of twenty-four unless the review says otherwise.

How it works

It opens the published app once for each role the brief needs, member, second member, instructor and admin, from saved sign-ins.
For each check it either reads the page and tests the text against the check's rule, or acts on the app: it creates a class with one place, books it as one member and tries it as another, downloads the export and reads its header, or follows the checkout to the payment page.
The last check matches the ten seed facts and flags invented text, such as a made-up number of sessions or testimonials nobody wrote.
A check that needs judgement, such as the look of a screen or a password reset email, takes the result I entered by hand and is marked as such.
It cleans up the bookings it made, writes one result file with a pass or fail and a note per check, and takes a screenshot of every check.
After each of the twelve changes the whole list runs again, and the results are compared with the previous pass so a change that broke something is caught.

What it records

Per check: Passed or failed, a note on what was found, whether the result came from a person, and a screenshot.
Per run: The app address, the tag of the build, the time, and the count passed out of the total.

What it cannot catch

Each builder's app has its own sign-in and page structure, so the checker is adapted per builder, and for many reviews the checks were run by hand with the same list.
Checks that need the reader's own keys, payments and emails, are left out and the grade is taken in proportion.
Invented content is caught by fixed patterns; dead buttons and placeholder data are found by using the app.

Where it was used

Every published review that ran this checker, with the figure it produced there, read from the review’s own data. Each figure opens the test results behind it.

Reviews that used the spec check
Review	Figure	Tested
Adalo	12 / 24 checks passed before the changes	Invalid Date
Anything	23 / 24 checks passed	Invalid Date
Atoms	22 / 24 checks passed	Invalid Date
Base44	30 / 30 checks passed	Invalid Date
Blink.new	22 / 24 checks passed	Invalid Date
Bolt.new	23 / 24 checks passed	Invalid Date
Bubble	21 / 24 checks passed	Invalid Date
Draftbit	24 / 24 checks passed, of those run	Invalid Date
Dreamflow	10 / 24 checks passed	Invalid Date
Emergent	26 / 26 checks passed, of those run	Invalid Date
Famous.ai	24 / 24 checks passed	Invalid Date
FlutterFlow	2 / 24 checks passed	Invalid Date
Noloco	20 of 24 checks passed	27 Oct 2001
RapidNative	10 / 24 checks passed, of those run	Invalid Date
Replit	28 / 30 checks passed	Invalid Date
rocket.new	22 / 24 checks passed, of those run	Invalid Date
Rork	15 / 24 checks passed	Invalid Date
Softgen	15 / 22 checks passed	Invalid Date
Softr	8 / 24 checks passed	Invalid Date
Zite	22 / 24 checks passed	Invalid Date