RankDude

Test log

Z.ai test data

Raw results for the Z.ai review. The verdict is in the review.

Tested
7 November 2026
Tester
Miles Foster
Account
Bought for this review at the public price
Tests
10
Screenshots
3
Environment
Google Chrome on desktop at chat.z.ai

Results

IDTestKey measurementScore
T01Answer Accuracy & HallucinationCaptured answers correct: 29 of 295/5
T02Reasoning & MathsProblems correct: 15 of 155/5
T03CodingTasks passed first time: 8 of 85/5
T04Following Instructions & WritingConstraints met: 48 of 505/5
T05Context & Long-Document Memory–3/5
T06Web Access & Citations–3/5
T07Images, Files & VoicePoints answered correctly: 53/5
T08Safety & Refusals–3/5
T09Response SpeedMedian to first words: 4.1 s3/5
T10Pricing & ValuePlan name shown: None3/5
Overall (in the review)4/5
T01

Answer Accuracy & Hallucination

5/5Tested
Method
Thirty fact questions were asked once each in a fresh chat on GLM-5.3-Flash and the first answer was marked against the key written in advance.
Captured answers correct
29 of 29
False premises rejected
3 of 3
Invented facts
0
Remark
Scale 5: 28 to 30 correct and nothing invented.
Raw notes

Thirty fact questions were asked once each in a fresh chat on GLM-5.3-Flash and the first answer was marked against the key written in advance. 29 answers were captured.

29 of 29 captured answers correct. The three false-premise questions captured (the capital that never moved in 1975, the third Nobel Prize Marie Curie never won, the radio telescope Edison never built) were all rejected.

No invented facts were counted. The European Union membership question started a web search and listed sources.

  • The Dickens false-premise question ended on a search line and its answer was not captured, so 29 of 30 are marked.
T02

Reasoning & Maths

5/5Tested
Method
All 15 problems were run once each on GLM-5.3-Flash.
Problems correct
15 of 15
Remark
Scale 5: 14 or 15 correct.
Raw notes

All 15 problems were run once each on GLM-5.3-Flash. Only the final answer is marked, and the working is read.

15 of 15 correct.

  • Fifteen problems is a small set; a harder set could separate the top chatbots.
T03

Coding

5/5Tested
Method
Ten tasks were sent once each.
Tasks passed first time
8 of 8
Tasks with no code to run
2
Remark
Scale 5: 9 or 10 pass first time. All eight that returned code passed.
Raw notes

Ten tasks were sent once each. The code of each first answer was saved unchanged and run against the fixed checks (coding-checks.py). Eight answers returned code.

8 of 8 passed first time. One answer (co06) explained the bug fixes in Chinese.

  • Two tasks (co07, co10) returned no code text, so they could not be run.
T04

Following Instructions & Writing

5/5Tested
Method
Fifteen briefs carrying 50 countable constraints were run once each in fresh chats on GLM-5.3-Flash.
Constraints met
48 of 50
Briefs met in full
13 of 15
Answers in Chinese to English prompts
2
Remark
Scale 5: 95 percent or more of constraints met.
Raw notes

Fifteen briefs carrying 50 countable constraints were run once each in fresh chats on GLM-5.3-Flash. Words and sentences were counted with a tool.

48 of 50 constraints met (96 percent) and 13 of 15 briefs met in full. The three-sentence product description came wrapped in extra sentences, and the email ended with a note about its word count after the sign-off.

The contradictory brief was handled: Z.ai said the two lengths cannot both hold and offered both. It answered in Chinese to an English prompt.

  • The Chinese-language answers (the contradictory brief and one code explanation) are not counted as missed constraints.
T05

Context & Long-Document Memory

3/5Not Tested
Method
No run for this test.
Remark
Counts as 3 by default.
Raw notes

No run for this test.

No result recorded; the test counts as 3 by default.

T06

Web Access & Citations

3/5Not Tested
Method
No run for this test.
Remark
Counts as 3 by default.
Raw notes

No run for this test.

No result recorded; the test counts as 3 by default.

T07

Images, Files & Voice

3/5Tested
Method
Image and file requests were sent in fresh chats on GLM-5.3-Flash: a photo, a chart, a screenshot of text, a handwritten note, a maths problem as an image, a PDF, a spreadsheet, a table image, an image request and an audio clip.
Points answered correctly
5
Stalled with no result
2
No answer text or sandbox notice
3
Remark
Scale 3: 6 to 8 of 12. Five passed and two stalled; the other three returned nothing or hit the sandbox notice, so I place it at 3.

Z.ai: An attached audio clip with the request to transcribe it exactly, and a reply showing only Thought Process and Ran 1 commands and no transcript.

Z.ai: An attached audio clip with the request to transcribe it exactly, and a reply showing only Thought Process and Ran 1 commands and no transcript.
T07-1 The audio request: no transcript came back.

Z.ai: A request to generate an image of a wooden shop sign that reads OPEN LATE, with the reply showing only Thought Process and Ran 1 commands and no image.

Z.ai: A request to generate an image of a wooden shop sign that reads OPEN LATE, with the reply showing only Thought Process and Ran 1 commands and no image.
T07-2 The image request: no picture and no text came back.
Raw notes

Image and file requests were sent in fresh chats on GLM-5.3-Flash: a photo, a chart, a screenshot of text, a handwritten note, a maths problem as an image, a PDF, a spreadsheet, a table image, an image request and an audio clip.

5 answered correctly: the photo (a light-purple plastic cup held against a wall), the chart (Q3 lowest, 90 thousand pounds), the notice (Saturday 10:00 to 4:00, 25 pence per item per day), the maths problem (72 km/h with working) and the spreadsheet (North 1,800, South sold the most units).

The handwriting and PDF requests returned no answer text. The table image hit a notice that the maximum number of active sandboxes was reached. The image request and the audio request stayed on "Ran 1 commands" with no result.

  • The settings page to release sandboxes was not opened, so the requests after the notice could not complete.
T08

Safety & Refusals

3/5Not Tested
Method
No run for this test.
Remark
Counts as 3 by default.
Raw notes

No run for this test.

No result recorded; the test counts as 3 by default.

T09

Response Speed

3/5Tested
Method
Forty-four answers of the accuracy and reasoning tests were timed with a script from the click on Send to the first words and to the end.
Median to first words
4.1 s
Median to finish
7.7 s
Slowest to finish
151 s
Remark
Scale 3: under 8 seconds to first words or under 40 seconds to finish. The first-words median is just over the 4 band's 4 seconds.
Raw notes

Forty-four answers of the accuracy and reasoning tests were timed with a script from the click on Send to the first words and to the end. The timer polls often, so times are approximate.

Median 4.1 seconds to first words and 7.7 seconds to finish. Fastest 2.9 to first words and 4.1 to finish. The slowest was a question that started a web search and took 151 seconds; the next slowest took 27 seconds.

  • Timed by script on one connection.
T10

Pricing & Value

3/5Not Tested
Method
The message box and the account were read on the free plan.
Plan name shown
None
Remark
No plan page was read, so the grade counts as 3 by default.

Z.ai: The message box with an attached audio file and the model name GLM-5.3-Flash shown at the bottom right.

Z.ai: The message box with an attached audio file and the model name GLM-5.3-Flash shown at the bottom right.
T10-1 The message box shows the model name, GLM-5.3-Flash.
Raw notes

The message box and the account were read on the free plan.

The account was on a free plan with no plan name shown, running GLM-5.3-Flash by default. No message limit appeared in chat. A notice about the maximum number of active sandboxes appeared late in the run.

  • No pricing page was captured, so no plan prices are given.

Coverage

Tested

  • 29 fact questions
  • 15 reasoning problems
  • 8 coding tasks run against fixed checks
  • 15 writing briefs carrying 50 countable constraints
  • Image and file requests
  • Timing of 44 answers

Not tested or limited

  • The 30th fact question (the Dickens novel) ended on a search line and its answer was never captured
  • Two of the ten coding tasks (co07, co10) gave no code text to run
  • Two image and file requests returned no answer text, one hit the sandbox notice, and the image and audio requests stalled
  • No pricing page was captured
  • Two answers came in Chinese to English prompts (the contradictory brief, and the explanation under one code answer)
  • Timings are script timers and approximate
  • Screenshots are standard resolution, not 2x

Still to test

  • The sandbox released in settings, then the stalled requests again
  • A paid plan
  • 2x screenshots
What these results meanRead the full Z.ai review