Test log
UX Pilot test data
Raw results for the UX Pilot review. The verdict is in the review.
- Tested
- 5 November 2026
- Tester
- Miles Foster
- Account
- Free demo account, signed up by RankDude, nothing paid to the brand
- Tests
- 9
- Screenshots
- 7
- Method
- Published methodology
- Environment
- Google Chrome on desktop
Results
| ID | Test | Key measurement | Score |
|---|---|---|---|
| T01 | Brand Kit & Consistency | – | n/a |
| T02 | Sharing & Collaboration | – | n/a |
| T03 | Brief to First Draft | Brief to draft: 76 seconds | 5/5 |
| T04 | Brief Adherence | Required elements right: 55 of 60 | 2/5 |
| T05 | First Draft Quality | Design faults: 8 | 3/5 |
| T06 | Editing Freedom | Edits that worked: 6 of 7 tried | 4/5 |
| T07 | Export & Handoff | Export points: 3 of 10 | 2/5 |
| T08 | Rights & Commercial Use | Free plan use: Personal only | 3/5 |
| T09 | Pricing & Value | Pro, paid yearly: $19 a month | 3/5 |
| Overall (in the review) | 3.7/5 | ||
Brand Kit & Consistency
n/aNot Tested- Method
- Upload logo, fonts and colours as a brand kit, then generate ten designs and check them.
- Remark
- Not graded; left out of the average.
Raw notes
Upload logo, fonts and colours as a brand kit, then generate ten designs and check them.
Not run: ten more designs do not fit the 80 credits a day on the free plan, and the three briefs and seven edits had used 62 of them.
- Needs a day's credits on a paid plan.
Sharing & Collaboration
n/aNot Tested- Method
- Eight points tried with a second account.
- Remark
- Not graded; left out of the average.
Raw notes
Eight points tried with a second account.
Not run: seats, roles and real-time collaboration are listed on the Teams plan, and no second account was made.
- Needs a Teams plan and a second test account.
Brief to First Draft
5/5Tested- Method
- Each brief was typed into the chat with the supplied files attached.
- Brief to draft
- 76 seconds
- Remark
- Scale 5: median under 5 minutes.
Raw notes
Each brief was typed into the chat with the supplied files attached. The clock ran from pressing send to the moment the chat stopped working, read from a script that watched the page and from screenshots every 10 seconds.
The bakery set finished in 73 seconds, the landing page in 76 seconds and the booking app in 143 seconds. The median is 76 seconds.
Text and shapes were editable on the canvas, not a picture.
- The time to the first screen was not clocked separately.
Brief Adherence
2/5Tested- Method
- Twenty required elements per brief were checked in the first draft before any editing: text, fonts, colours and image use read from the page code of each screen, sizes from the frame and the card inside it, the rest by eye at 2x.
- Required elements right
- 55 of 60
- Invented items
- 9
- Remark
- Scale 2: invented claims, here "Locally sourced & carefully packed". By count alone, 55 of 60 would be 4.
Raw notes
Twenty required elements per brief were checked in the first draft before any editing: text, fonts, colours and image use read from the page code of each screen, sizes from the frame and the card inside it, the rest by eye at 2x.
The landing page had 20 of 20 right. The bakery set had 18 of 20: the post card is smaller than 1080 by 1080 and the story card is 450 by 800, not 1080 by 1920. The banner card is 820 by 312 as asked. The booking app had 17 of 20: screens are 375 pixels wide, not 390, the session cards say "60 min session" and "8 hours coverage" instead of "60 min" and "8 hours", and the studio photo sits in a small box, not as the Welcome background. 55 of 60 in all. Punctuation alone, such as a full stop added or dropped at the end of a line, was not counted as a miss in any of the three briefs.
Invented items: 9. On the landing page, a postcode line ("London, OR1 2PW"), a "Book a spot" button, a "Follow our making" block with social icons, the links Shipping, Privacy and Returns, the line "Locally sourced & carefully packed" and three AI-made product photos. On the app, the month October 2023, the booking summary on the last screen and the extra buttons Continue and Done. Counted as nine: the postcode line, Book a spot, Follow our making, the Shipping, Privacy and Returns links, the Locally sourced line, the AI product photos, October 2023, the booking summary, and the Continue and Done buttons.
- Sizes are read from the card inside each frame; the post card's exact size was not saved as a measurement, only that it is smaller than 1080 by 1080. Every frame is 1440 by 1024 pixels whatever the brief asked for.
First Draft Quality
3/5Tested- Method
- The first drafts were opened at their sizes and checked for contrast, overlap and faint or hidden images.
- Design faults
- 8
- Remark
- Scale 3: 6 to 10 faults.
Raw notes
The first drafts were opened at their sizes and checked for contrast, overlap and faint or hidden images. Contrast was calculated from the hex values (WCAG AA: 4.5, large text 3). The landing page was also opened at 360 and 768 pixels wide.
8 faults. Bakery 4: the orange $18 on peach is 1.9 to 1, the cream Order now label on orange is 2.25 to 1, the story headline sits on decorative circles, and the banner headline runs into the faded photo panel. Booking app 4: the gold prices on white are 2.4 to 1, the grey session lines are 2.5 to 1, the calendar's weekday labels are under 2.6 to 1 and the grey days of the last month are 1.24 to 1.
The landing page had no fault I could measure: bark text on oat is 11.2 to 1, and at 360 and 768 pixels the hero stacks and the links fold into a menu.
- Tap target sizes were not measured. At 360 and 768 pixels only the top of the landing page was looked at.
Editing Freedom
4/5Tested- Method
- Seven of the twelve fixed edits were tried on the landing page: by hand in Manual Edit, and by a written instruction in the chat.
- Edits that worked
- 6 of 7 tried
- Not run
- 5
- Remark
- Scale 4: scaled from 6 of 7 tried to about 10 of 12.
Raw notes
Seven of the twelve fixed edits were tried on the landing page: by hand in Manual Edit, and by a written instruction in the chat. An edit counts when it did what was asked with a normal control and no workaround. The grade is scaled from the seven tried.
Six worked. Font: one instruction changed every headline to Playfair Display (49 seconds, 10 credits). One colour everywhere: one instruction changed every berry #A23B5B to #1F6F8B (18 seconds, 2 credits). Resize a text box: the width field in Manual Edit narrowed the paragraph once the change was saved. One word edited in place by double-clicking and typing. Canvas size: the width box set the page to 768 pixels and the layout stayed whole. One element regenerated from a written instruction: the hero button came back larger (35 seconds, 4 credits).
One did not: dragging the hero button never dropped and was cancelled. Not run: swap and crop an image, add a section, align three elements, undo three steps, lock an element. Changing the colour in Manual Edit's colour list changed the field but not the page.
- Five edits were not run, so this grade rests on seven.
- In the word-edit test the typed word landed inside the word 'packed' (the screenshot shows 'packetinyd') because of where the test clicked, and the next chat edit corrected it, so the control worked but the test's click was off. The regenerate instruction said 'Change nothing else', yet the chat's summary in the editing picture also corrected the hero spelling and changed the paragraph width
Export & Handoff
2/5Tested- Method
- Ten export points were tried on the finished designs: the export panel, a PDF of one screen and a PDF of all nine screens, opened in a viewer and read for text, page size and any badge.
- Export points
- 3 of 10
- Remark
- Scale 2: 3 to 5 of 10.
Raw notes
Ten export points were tried on the finished designs: the export panel, a PDF of one screen and a PDF of all nine screens, opened in a viewer and read for text, page size and any badge.
Three points worked: the PDF is sharp with real text and nothing is upscaled, it names its fonts (Fraunces and Inter are embedded in the file), and all nine screens came down in one nine-page PDF. The other seven did not: there is no PNG download (copy as image only), no SVG, no PDF with bleed, Figma and code (HTML, React, ZIP) are locked, a "Made with UX Pilot" strip sits at the foot of every page, and my uploaded files cannot be exported.
The code viewer shows the first lines of main.html behind a "See full code" button.
- Paid exports were not tried: no plan was bought.
Rights & Commercial Use
3/5Tested- Method
- I read the Terms of Service and the Privacy Policy from the site footer and looked for a training setting.
- Free plan use
- Personal only
- Indemnity for output
- None
- Remark
- Scale 3: commercial use only on paid plans.
Raw notes
I read the Terms of Service and the Privacy Policy from the site footer and looked for a training setting.
On the free plan the terms say "Free plan users may only use generated designs for personal, non-commercial purposes." Paid plans get a licence: "a non-exclusive, non-transferable license to use generated designs for commercial purposes". UX Pilot keeps the rights in its service, so the terms give you a licence rather than ownership.
On training the terms say "We do not train our AI models using your generated designs or any confidential information." There is no indemnity for generated content, and I found no training switch.
- The terms name a Standard plan that the pricing page does not list.
Pricing & Value
3/5Tested- Method
- Plans were read on the public pricing page in both billing views and in the product's own credits panel.
- Pro, paid yearly
- $19 a month
- Pro, monthly
- $29
- Remark
- Scale 3: $200 to $299 a year.
Raw notes
Plans were read on the public pricing page in both billing views and in the product's own credits panel.
Pro is the cheapest plan with commercial rights, export to Figma and code and 750 credits a month: $19 a month paid yearly, which is $228 a year, or $29 a month paid monthly. Teams is $31 a user a month paid yearly, or $39 monthly.
Free gives 80 credits a day with no rollover, 13 screens a day and one project. My three briefs cost 46 of the 80 credits (the first design was free) and seven edits cost 16 more.
- No plan was bought; renewal prices were not seen.
Other features checked
Coverage
Tested
- The three fixed briefs typed into the chat, with the supplied logo and photos attached
- Twenty required elements per brief checked in the first draft, read from the page code, with contrast calculated
- Seven of the twelve fixed edits, by hand and by written instruction
- The export panel, a PDF export of one screen and of all nine
- The Terms of Service, the Privacy Policy and the public pricing page read
Not tested or limited
- All three briefs ran in one project, because the free plan allows one project. Each brief started a new chat thread, but the bakery set and the booking app screens were already on the canvas
- The bakery brief was the account's first design and cost no credits; the other briefs were charged, so the credits per brief are not like for like
- The pieces are web screens: the bakery post, story and banner came out as cards inside 1440 by 1024 pixel frames, and the app screens are 375 pixels wide
- Seven of the twelve fixed edits were tried. Swap and crop an image, add a section, align three elements, undo three steps and lock an element were not run; the editing grade is scaled from the seven tried
- A drag of the hero button never dropped and was cancelled, so moving an element by hand counts as not done
- In the word-edit test the typed word landed inside the word 'packed' (the screenshot shows 'packetinyd') because of where the test clicked, and the next chat edit corrected it, so the control worked but the test's click was off. The regenerate instruction said 'Change nothing else', yet the chat's summary in the editing picture also corrected the hero spelling and changed the paragraph width
- Brand kit was not graded: ten more designs do not fit the 80 credits a day. Collaboration was not graded: sharing and roles are on the Teams plan
- The pricing page names Pro and Teams; the terms also name a Standard plan that I did not see on the page
Still to test
- The five edits not run
- Exports on Pro
- Brand kit and ten designs on a paid day
- Collaboration with a second account on Teams