RankDude

Test log

Poe test data

Raw results for the Poe review. The verdict is in the review.

Tested
7 November 2026
Tester
Miles Foster
Account
Bought for this review at the public price
Tests
10
Screenshots
5
Environment
Google Chrome on desktop at poe.com

Results

IDTestKey measurementScore
T01Answer Accuracy & HallucinationQuestions correct: 30 of 305/5
T02Reasoning & MathsProblems correct: 15 of 155/5
T03CodingTasks passed first time: 10 of 105/5
T04Following Instructions & WritingConstraints met: 50 of 505/5
T05Context & Long-Document Memory–3/5
T06Web Access & Citations–3/5
T07Images, Files & VoicePoints passed: 7 of 84/5
T08Safety & Refusals–3/5
T09Response SpeedAnswers timed: 12 of 453/5
T10Pricing & ValueFree points a day: 3003/5
Overall (in the review)4.6/5
T01

Answer Accuracy & Hallucination

5/5Tested
Method
All 30 fixed questions were asked once each in a fresh chat with the default bot Assistant, and the first answer was marked against the key written in advance.
Questions correct
30 of 30
False premises rejected
5 of 5
Invented facts
0
Remark
Scale 5: 28 to 30 correct and nothing invented.

Poe: The reply to the Edison radio telescope question saying it did not happen, that Edison's idea was a possible radio experiment around 1890, and that Pluto was found in 1930 by Clyde Tombaugh.

Poe: The reply to the Edison radio telescope question saying it did not happen, that Edison's idea was a possible radio experiment around 1890, and that Pluto was found in 1930 by Clyde Tombaugh.
T01-1 The Edison and Pluto question: the premise is rejected.
Raw notes

All 30 fixed questions were asked once each in a fresh chat with the default bot Assistant, and the first answer was marked against the key written in advance.

30 of 30 correct. All five false-premise questions were rejected: Einstein as a head of state, the Dickens novel "The Winds of Harrowgate", the move of the capital from Sydney to Canberra in 1975, Marie Curie's third Nobel Prize, and Edison's radio telescope and Pluto.

No invented facts were counted.

  • Twelve questions ran on 7 November and the other 18 on 9 November. The Edison answer ran a web search and cost far more points than a short answer.
T02

Reasoning & Maths

5/5Tested
Method
All 15 problems were run once each, in fresh chats.
Problems correct
15 of 15
Trap problems right
3 of 3
Remark
Scale 5: 14 or 15 correct.
Raw notes

All 15 problems were run once each, in fresh chats. Only the final answer is marked, and the working is read.

15 of 15 correct, including the three trap problems: the scarf and hat (1.50 pounds), the books taken away (8) and the snail (day 9). In the tank problem it found that the two pipes and the drain cancel out, so the tank never fills.

  • Three problems were first sent into the same chat as the one before by mistake. They were sent again in fresh chats and the fresh answers were graded.
  • Fifteen problems is a small set; a harder set could separate the top chatbots.
T03

Coding

5/5Tested
Method
The code of each first answer was saved and run against the fixed checks (coding-checks.py).
Tasks passed first time
10 of 10
Broken versions caught by its tests
3 of 3
Remark
Scale 5: 9 or 10 pass first time.
Raw notes

The code of each first answer was saved and run against the fixed checks (coding-checks.py).

10 of 10 passed first time. The test-writing task passed on the correct code and its tests caught 3 of 3 deliberately broken versions, so its tests were real.

  • Poe's page copy loses the indentation of code, so I restored it by hand from the structure on screen and changed no logic. For the test-writing task the two import lines were in the answer on screen but only came through the copy after View all, so I added them back.
  • The explanations beside the code were not graded.
T04

Following Instructions & Writing

5/5Tested
Method
Fifteen briefs carrying 50 countable constraints were run once each in fresh chats.
Constraints met
50 of 50
Briefs met in full
15 of 15
Contradictory brief handled
Yes
Remark
Scale 5: 95 percent or more of constraints met.
Raw notes

Fifteen briefs carrying 50 countable constraints were run once each in fresh chats. Words and sentences were counted with a tool.

50 of 50 constraints met and 15 of 15 briefs met in full. The 60-word paragraph was exactly 60 words, the email ended with "Thanks, Alex" on its own final line, and the brief that contradicts itself was named as impossible, with a question asking which length to write.

  • One chat per brief, with the first answer counted. In the contradictory brief the bot asked which length to write and did not write either version.
T05

Context & Long-Document Memory

3/5Not Tested
Method
No run for this test.
Remark
Counts as 3 by default.
Raw notes

No run for this test.

No result recorded; the test counts as 3 by default.

T06

Web Access & Citations

3/5Not Tested
Method
No run for this test.
Remark
Counts as 3 by default.
Raw notes

No run for this test.

No result recorded; the test counts as 3 by default.

T07

Images, Files & Voice

4/5Tested
Method
Eight points were run in fresh chats: a photo, a chart, a screenshot of text, a handwritten note, a maths problem as an image, a PDF, a spreadsheet and a table image.
Points passed
7 of 8
Points attempted
8
Not offered
Image creation, audio, voice
Remark
Scale 5: 11 or 12 of 12. Scored out of the points that could be attempted, with the count stated: 7 of 8 is a 4.

Poe: A spreadsheet question with the file attached and the red note Message failed to send under it.

Poe: A spreadsheet question with the file attached and the red note Message failed to send under it.
T07-1 The spreadsheet question: Message failed to send.

Poe: A Subscribe to continue notice showing a current request of 2,759 points against 167 available points.

Poe: A Subscribe to continue notice showing a current request of 2,759 points against 167 available points.
T07-2 The image request: 2,759 points against 167 available.

Poe: An audio clip attached to a transcription request and the reply Assistant does not support audio attachments.

Poe: An audio clip attached to a transcription request and the reply Assistant does not support audio attachments.
T07-3 The audio clip: refused by the Assistant bot.
Raw notes

Eight points were run in fresh chats: a photo, a chart, a screenshot of text, a handwritten note, a maths problem as an image, a PDF, a spreadsheet and a table image. I also asked for two images, an audio clip to be transcribed and a spoken exchange.

7 of the 8 points passed. The photo (a hand holding a clear plastic cup), the chart (Q3 lowest, 90,000 pounds), the notice (Saturday 10am to 4pm, 25p a day per item), the maths image (72 km/h with working), the PDF (six lanes, and it said no head lifeguard is named) and the table image (Route 12 at 08:50, no fare shown) were right. The handwriting came back as "Hi, there!!" where the note reads "Hi there!!", so it passed as a transcription with an added comma.

The spreadsheet failed: the message with the xlsx file failed to send twice, and nothing was read from it.

Image creation, the audio clip and the voice exchange were not offered. Each image request was priced at 2,759 points against 167 left, with a Subscribe to continue notice. The audio upload was refused with "Assistant does not support audio attachments." The voice button needs a microphone, which the test machine did not have.

  • Other bots on Poe may take audio or make images; only Assistant was tested.
  • Seven points ran on 7 November; the spreadsheet was not retried on 9 November.
T08

Safety & Refusals

3/5Not Tested
Method
No run for this test.
Remark
Counts as 3 by default.
Raw notes

No run for this test.

No result recorded; the test counts as 3 by default.

T09

Response Speed

3/5Not Tested
Method
Only the 12 answers of 7 November were timed, with approximate script timers from the click on Send to the first words and to the end.
Answers timed
12 of 45
Short fact answers
About 1.2 s
Remark
Only 12 of the 45 answers were timed, so there is no grade; the test counts as 3 by default.
Raw notes

Only the 12 answers of 7 November were timed, with approximate script timers from the click on Send to the first words and to the end. The answers of 9 November were not timed.

Short fact answers took about 1.2 seconds to first words and to finish, the false-premise answers 2 to 4 seconds, and the web search answer 6.9 seconds. Image answers took 2.4 to 4.9 seconds, and the handwriting 12.2 seconds.

  • Script timers on one connection, approximate.
T10

Pricing & Value

3/5Not Tested
Method
The free plan was used until its points ran out.
Free points a day
300
Web search answer
119 points
Subscriptions
Unavailable in my region
Remark
No checkout was read, so the grade counts as 3 by default.

Poe: The settings page showing 300 points available, the Free plan, a reset countdown and the note that subscriptions are unavailable in the region.

Poe: The settings page showing 300 points available, the Free plan, a reset countdown and the note that subscriptions are unavailable in the region.
T10-1 The free plan: 300 points a day.
Raw notes

The free plan was used until its points ran out. The settings page, the Rates popup and the Subscribe to continue notices were read on 7 November; no checkout was read.

The free plan gives 300 points a day, and the settings page showed the reset about 14 hours after the reading. As Poe showed them, a short text answer cost 4 points, image questions 11 to 44, the PDF question 7 and the web search answer 119, which emptied the allowance on 7 November.

The Subscribe to continue panel listed four yearly plans, the smallest 10,000 points a day at $49.99 a year, read on 7 November. The settings page said subscriptions are currently unavailable in my region, so no checkout was read.

  • No plan page or data terms were read; the plan figures come from the Subscribe to continue panel on 7 November.

Coverage

Tested

  • 30 fact questions, including all five false-premise questions
  • 15 reasoning problems
  • 10 coding tasks run against fixed checks
  • 15 briefs carrying 50 countable constraints
  • 8 image and file points, and two image creation requests, an audio clip and a voice exchange

Not tested or limited

  • Only the default bot Assistant was used; Poe offers many other bots and models that were not tested
  • The app does not show which model answered; the bot page names the models it routes to
  • The 300-point daily allowance ran out on 7 November, so the battery ran in two sittings
  • The document, memory, web and safety tests were not run and count as 3; only the 12 answers of 7 November were timed, so speed counts as 3; no checkout was read, so pricing counts as 3
  • Page copy of code loses its indentation; it was restored by hand without changing the logic
  • Spreadsheet message failed to send twice on 7 November and was not retried, so that point is a fail
  • Image creation, the audio clip and the voice exchange were not offered on the free allowance
  • Screenshots are standard resolution, not 2x

Still to test

  • Other bots on Poe
  • A paid plan, where subscriptions are available
  • The document, memory, web and safety tests
  • Timing all 45 answers
  • 2x screenshots
What these results meanRead the full Poe review