Test log
Replit test data
Raw results for the Replit review. The verdict is in the review.
- Tested
- 26 October 2026
- Tester
- Miles Foster
- Account
- Bought for this review at the public price
- Tests
- 9
- Screenshots
- 6
- Method
- Published methodology
- Environment
- Google Chrome on desktop
Results
| ID | Test | Key measurement | Score |
|---|---|---|---|
| T01 | Prompt to Working App | Brief to public address, wall clock: 11 h 13 min, 3 messages | 2/5 |
| T02 | Spec Accuracy | Checks passed: 28 of 30 | 4/5 |
| T03 | Sign-in & Access Control | Attacks blocked: 2 of 2 | 5/5 |
| T04 | Database & Admin | Databases: Development and Production, separate | 5/5 |
| T05 | Payments & Email Actions | Checkout, order and credits: passed | 4/5 |
| T06 | Iteration & Credit Burn | Worked in one message: 10 of 12 | 4/5 |
| T07 | Publish, Speed & Mobile | First publish: about 7 min (after Stripe was removed) | 2/5 |
| T08 | Export & Ownership | Files in the download: 910 | 3/5 |
| T09 | Pricing & Cost to Finish | Credits used: about $5 of $20 (before the Stripe removal) | 4/5 |
| Overall (in the review) | 3.7/5 | ||
Prompt to Working App
2/5Tested- Method
- The Fitting Room brief was pasted word for word (2,236 characters, Power mode) and sent at 20:22 UTC on 25 October 2026.
- Brief to public address, wall clock
- 11 h 13 min, 3 messages
- Brief to first version, wall clock
- 64 min, 2 messages
- Agent working time
- about 7 min build, 7 min Stripe removal
- Remark
- By the published scale the clock stops at a public address with 25 of 30 checks passing. The address was reached 11 h 13 min after the brief, under 12 hours: 2/5. Most of that was waiting for a decision about Stripe, not Replit working: the first version, with 28 passing, was ready in 64 minutes and the Agent's own work for the build and the Stripe removal was about 14 minutes. The owner decides how to read it.
Raw notes
The Fitting Room brief was pasted word for word (2,236 characters, Power mode) and sent at 20:22 UTC on 25 October 2026. The clock stops when the app is live at a public address and 25 of the 30 checks pass.
The Agent worked about 7 minutes, plus short turns of 37, 15 and 24 seconds. It paused 37 seconds in to ask me to connect Stripe, and the test sat idle about 10 minutes (not Replit's time). It asked twice for the owner's Resend key (declined) and asked which email should be the first admin.
The Agent reported its first build done at 21:26 UTC, 64 minutes after the send on the wall clock, with 1.95 dollars of credits used. The first admin had to sign up and verify before the Agent could grant admin: that was a second message. The first version passed 28 of 30 checks on the private preview.
The first version could not be published while it used Stripe. After the Stripe test account was claimed, Replit asked for a live Stripe account and I stopped. On 26 October one more message removed the payment feature (the Agent worked 7 minutes), I deleted the sandbox connection by hand, and the app answered at https://fitting-room-pilates.replit.app/ at 07:35 UTC, 11 hours 13 minutes after the brief. Three messages in all (build, first admin, remove Stripe).
- The idle time waiting for the Stripe connect and for the decision to remove Stripe was my test sitting still, not Replit's. The 30 checks were not run again on the published app: the 28 is from the first version.
Spec Accuracy
4/5Tested- Method
- The 30 checks were run on the app's private preview, by hand and by script, with a screenshot per check.
- Checks passed
- 28 of 30
- Seed facts correct
- 10 of 10
- Invented items
- 1
- Remark
- 28 passing is in the 28 to 30 band, but one invention means not 5: 4/5, the same ruling as the Base44 review.
Raw notes
The 30 checks were run on the app's private preview, by hand and by script, with a screenshot per check. I also counted anything the brief never asked for.
28 of 30 passed on the first version. The two that failed: C28 and C29, the booking and admin emails, because Replit needs the owner's own Resend account and key and none was given. The app saved the booking and said delivery was pending setup; eight emails sat as pending in the admin screen.
All ten seed facts were correct. One invention: a line saying credits come back if you cancel before class starts, a rule the brief never set.
- Checks ran on the private preview, not a public address.
Sign-in & Access Control
5/5Tested- Method
- Sign-up, log-in, log-out and password reset for a member, an instructor and an admin, then two attacks with the wrong session.
- Attacks blocked
- 2 of 2
- Password reset
- worked
- Remark
- Every flow works and both attacks are blocked: 5/5.
Raw notes
Sign-up, log-in, log-out and password reset for a member, an instructor and an admin, then two attacks with the wrong session. Requests and responses are saved as transcripts.
Sign-up with a six-digit emailed code, log-in, log-out and password reset by code all worked for every role. Attack 1 was blocked: filters by member B's id or email returned only A's rows, and changing or deleting B's booking gave 'Booking not found' or 403; B's booking was unchanged. Attack 2 was blocked: a member calling the admin routes got 403 'requires an approved studio role' for summary, class delete, class edit and the CSV; all classes were still there.
- Replit's own security scan was not run. The attacks ran against the private preview with copies of my own session.
Database & Admin
5/5Tested- Method
- Six checks: records written, kept after a reload, a republish and a design change; the admin can view, filter, edit and delete in the tool; records export as a file.
- Databases
- Development and Production, separate
- Production tables after publish
- 8, with their rows
- Checks passed
- 6 of 6
- Remark
- All six checks passed: 5/5.
Raw notes
Six checks: records written, kept after a reload, a republish and a design change; the admin can view, filter, edit and delete in the tool; records export as a file. After publishing I opened Replit's Database tool.
Bookings, orders, classes and form messages were saved, and the admin could see, filter, add, change and delete them in the app; the bookings CSV export worked. After the dark mode change the Agent reported the same classes, bookings and orders before and after.
The Database tool shows two separate databases, Development and Production, each with its tables and row counts and a My Data editor. After I published, Production held the test records: bookings 4, checkouts 4, classes 31, emails 8, inquiries 1, orders 2, users 4 and waitlist 0.
- I did not edit or delete a row in the My Data editor itself: the admin checks ran in the app's own admin screens. The data was read in Production once, after the publish.
Payments & Email Actions
4/5Tested- Method
- A Stripe test checkout, an order and credits, a declined card and the booking and admin emails, run on the first version before Stripe was removed.
- Checkout, order and credits
- passed
- Declined card
- passed
- Emails sent
- 0
- Remark
- Three of the four checks passed on the first version: 4/5. The email check did not pass because no key was given, not because the tool could not do it. The payment feature was removed afterwards and is not part of the published app.
Raw notes
A Stripe test checkout, an order and credits, a declined card and the booking and admin emails, run on the first version before Stripe was removed.
Replit set up the Stripe test connection itself. A $95 pass and a $24 single class both completed in test mode; the admin's Orders tab listed both and the credits were right (5 and 1). The declined card 4000 0000 0000 0002 showed 'Your credit card was declined', and made no order and no credits. Checkout showed shekels by default for a visitor in my location, with a US dollar choice.
The emails were not sent. Replit needs the owner's own Resend account and key, and none was given: the app stored them as pending. The Agent asked for the key twice more during change 8.
I then removed the payment feature with one message (the Agent worked 7 minutes). It reported the Passes page, checkout, orders, APIs, Stripe packages and webhooks removed, 5 free credits once for each new member, existing data unchanged and booking, cancel, capacity and waiting list passing its own tests, with one manual step left, which I did. Payments are not part of the published app.
- I declined the Resend connection on purpose, as I would not hand over the owner's key. Webhook confirmation was read from the order and credits appearing, not from the Stripe event log.
Iteration & Credit Burn
4/5Tested- Method
- The twelve scripted changes, one at a time, each one message.
- Worked in one message
- 10 of 12
- Regressions found
- 0
- Credits for the twelve
- about $2.80
- Remark
- 9 or 10 first try with no regression found is 4/5.
The twelve changes
| # | Change | Result | Agent worked |
|---|---|---|---|
| 1 | Phone field on sign-up | Done, one message | not read |
| 2 | Rename instructor to coach | Done, one message | 2 min |
| 3 | Dark green and cream colours | Done, one message | 1 min |
| 4 | An About page | Done, one message | 1 min |
| 5 | Power Reformer capacity 4 | Done, one message | 2 min |
| 6 | Coach filter on the timetable | Done, one message | 1 min |
| 7 | Planted bug, then fix | Fix took a second message, as planned | 1 min each |
| 8 | Second email when admin cancels | Built, but cannot send without a Resend key | 5 min |
| 9 | Phone layout, one class per row | Sent, result not recorded | not read |
| 10 | Bookings export for admin only | Already true, nothing changed | 51 s |
| 11 | Waiting list with position | Done, one message | 3 min |
| 12 | Dark mode toggle | Done, one message | 2 min |
1
- Change
- Phone field on sign-up
- Result
- Done, one message
- Agent worked
- not read
2
- Change
- Rename instructor to coach
- Result
- Done, one message
- Agent worked
- 2 min
3
- Change
- Dark green and cream colours
- Result
- Done, one message
- Agent worked
- 1 min
4
- Change
- An About page
- Result
- Done, one message
- Agent worked
- 1 min
5
- Change
- Power Reformer capacity 4
- Result
- Done, one message
- Agent worked
- 2 min
6
- Change
- Coach filter on the timetable
- Result
- Done, one message
- Agent worked
- 1 min
7
- Change
- Planted bug, then fix
- Result
- Fix took a second message, as planned
- Agent worked
- 1 min each
8
- Change
- Second email when admin cancels
- Result
- Built, but cannot send without a Resend key
- Agent worked
- 5 min
9
- Change
- Phone layout, one class per row
- Result
- Sent, result not recorded
- Agent worked
- not read
10
- Change
- Bookings export for admin only
- Result
- Already true, nothing changed
- Agent worked
- 51 s
11
- Change
- Waiting list with position
- Result
- Done, one message
- Agent worked
- 3 min
12
- Change
- Dark mode toggle
- Result
- Done, one message
- Agent worked
- 2 min
Raw notes
The twelve scripted changes, one at a time, each one message. After each change a short script checked the timetable, My bookings, passes, admin and instructor screens.
Ten of the twelve worked in one message (change 10 needed no change because it was already true). Change 7 planted a bug and the next message fixed it. Change 8 stalled: the Agent built the email records but needed a Resend key to send. Change 9 was sent but its result was not recorded, so it is not counted as done.
The twelve used about $2.80 of credits (from $2.12 to $4.92 on the usage page). Where I read the meter, a change cost between $0.08 and $0.58. The short check passed after every change that finished.
A thirteenth message, not one of the twelve, removed Stripe: the Agent worked 7 minutes. Its own browser sign-up test was blocked by human verification, so it said the signed-in screens were not fully verified in a browser. I did not run the checks again.
- Only a short check ran after each change, not all 30 checks again. Change 9 has no recorded result.
Publish, Speed & Mobile
2/5Tested- Method
- Publishing to the vendor address, Lighthouse on a phone three times on the home page.
- First publish
- about 7 min (after Stripe was removed)
- Median mobile score (3 runs, home page)
- 45
- Own domain
- not tried, none owned; Add a domain exists
- Remark
- Vendor address only, no domain tried, and a median of 45 falls in the 30 to 49 band, which is 2/5. The vendor address alone would be 3.
Raw notes
Publishing to the vendor address, Lighthouse on a phone three times on the home page. No test domain was connected because I own none for the test; layout counts and a real phone were not run.
Replit would not publish while the app used the Stripe test connection: the Publish button stayed disabled, and claiming the test account led to a request to activate a live Stripe account, where I stopped. After the Stripe feature was removed and the sandbox connection deleted in the Publishing pane, I pressed Publish at 07:28 UTC and the app answered with HTTP 200 at 07:35 UTC, about 7 minutes later, at https://fitting-room-pilates.replit.app/. It is public with no log-in wall.
Lighthouse mobile (version 13.5.0), three runs on the home page: 44, 45 and 45, median 45; largest paint 8.1 to 8.3 seconds, layout shift 0.31, blocking time 0 to 4 ms, about 1.2 MB in 27 requests.
I did not connect a custom domain, because I own none for the test. The Publishing pane has an Add a domain link next to the address.
- Only the home page was measured; layout faults at other widths, a real phone and add to home screen were not.
Export & Ownership
3/5Tested- Method
- The export is downloaded, unpacked, searched for every call that needs Replit, and listed.
- Files in the download
- 910
- Size
- 13.1 MB
- Remark
- The whole source leaves on this plan, but three Replit-provided services must be replaced by hand, and I read the zip without running it elsewhere: 3/5.
Raw notes
The export is downloaded, unpacked, searched for every call that needs Replit, and listed. It was not installed and started on another host.
The download at the project's .zip address works on Core, with no extra step: 910 files, 13.1 MB, a pnpm workspace. Replit couplings found by search: the connectors package (the Stripe key comes through Replit), stripe-replit-sync, Replit's Vite plugins, DATABASE_URL (a Postgres database from Replit) and Clerk keys (sign-in is provisioned and proxied by Replit). To run elsewhere you need your own Postgres, your own Clerk keys and a Stripe secret key, and the connectors call must be replaced. I did not read the terms.
- The zip was read, not installed and started against a database of my own.
Pricing & Cost to Finish
4/5Tested- Method
- Prices were read on replit.com/pricing on 26 October 2026.
- Credits used
- about $5 of $20 (before the Stripe removal)
- Credits left at last read
- $14.99
- Remark
- Finished within the entry paid plan with the allowance clear, but running cost is unknown: 4/5, the same ruling as the Bolt.new review.
What it cost to finish the test app
| Stage | Credits used |
|---|---|
| First working version | about $1.95 |
| Twelve changes | about $2.80 |
| Build and twelve changes | about $5 of $20 |
First working version
- Credits used
- about $1.95
Twelve changes
- Credits used
- about $2.80
Build and twelve changes
- Credits used
- about $5 of $20
Raw notes
Prices were read on replit.com/pricing on 26 October 2026. The credit ledger priced the whole test from the usage page.
Building and changing the test app used about $5 of the $20 of credits that come with Core (the usage page read $4.92 spent after the twelfth change). The first working version cost about $1.95 and the twelve changes about $2.80. Remaining credit last read: $14.99, before the Stripe removal. The Core plan receipt was $21.78, which is the $20 plan plus tax.
Publishing keeps using credits, and after the publish Replit offered to turn on auto-reload so the app is not interrupted; the billing page shows Auto-reload off. I did not read the usage page after the publish, so what the removal and the publish cost is unknown. Replit's pricing page also lists a pay-as-you-go option (effort-based pricing) that I did not read.
Stripe and Resend fees were not measured.
- Limit behaviour, the free plan's limits and the hosting cost after publishing were not read.
Coverage
Tested
- One brief built and checked with 30 written checks
- Two scripted attacks
- Stripe test checkout, a declined card and the emails (before Stripe was removed)
- Twelve changes with a short check after each
- Removing Stripe with one message, then publishing
- Phone speed on the published page, three runs
- The Database tool: development and production tables and row counts after publishing
- The project download, unpacked and searched for vendor couplings
- Plans, the credits used and the plan receipt
- The support route (a question sent through Get help) and the full cancellation
Not tested or limited
- The 30 checks ran on the first version, with Stripe, on the owner-only preview; they were not run again on the published app after Stripe was removed
- Emails were not sent because they need your own Resend key
- Change 9 (phone layout) was sent but its result was not recorded; change 8 could not finish without the Resend key
- Own domain was not tried because I own none for the test; layout faults at other widths and a real phone were not tested, and only the home page was measured for speed
- The export zip was unpacked and read, not run elsewhere, and the terms were not read
- The usage page was not read after the Stripe removal and publish, so their credit cost is unknown
- Support is not graded: my question was sent on 26 October 2026 and Replit has not replied yet
Still to test
- Own domain and layout faults (test 7)
- Running the download elsewhere (test 8)
- Support, graded once Replit replies
- Emails with a Resend key
- The 30 checks again on the published app