RankDude

Test log

Kimi test data

Raw results for the Kimi review. The verdict is in the review.

Tested
7 November 2026
Tester
Miles Foster
Account
Bought for this review at the public price
Tests
10
Screenshots
5
Environment
Google Chrome on desktop at kimi.ai

Results

IDTestKey measurementScore
T01Answer Accuracy & HallucinationQuestions correct: 30 of 305/5
T02Reasoning & MathsProblems correct: 15 of 155/5
T03CodingTasks passed first time: 10 of 105/5
T04Following Instructions & WritingConstraints met: 50 of 505/5
T05Context & Long-Document Memory–3/5
T06Web Access & Citations–3/5
T07Images, Files & VoicePoints passed: 8 of 85/5
T08Safety & Refusals–3/5
T09Response SpeedMedian to first words: 2.4 s4/5
T10Pricing & ValueBusy notices: Both days3/5
Overall (in the review)4.3/5
T01

Answer Accuracy & Hallucination

5/5Tested
Method
All 30 fixed questions were asked once each in a fresh chat in Instant High mode, and the first answer was marked against the key written in advance.
Questions correct
30 of 30
False premises rejected
5 of 5
Invented facts
0
Remark
Scale 5: 28 to 30 correct and nothing invented.
Raw notes

All 30 fixed questions were asked once each in a fresh chat in Instant High mode, and the first answer was marked against the key written in advance.

30 of 30 correct. All five false-premise questions were rejected: Einstein as a head of state, the Dickens novel "The Winds of Harrowgate", the move of the capital from Sydney to Canberra in 1975, Marie Curie's third Nobel Prize, and Edison's radio telescope and Pluto.

No invented facts were counted.

T02

Reasoning & Maths

5/5Tested
Method
All 15 problems were run once each in fresh chats.
Problems correct
15 of 15
Trap problems right
3 of 3
Remark
Scale 5: 14 or 15 correct.
Raw notes

All 15 problems were run once each in fresh chats. Only the final answer is marked, and the working is read.

15 of 15 correct, including the three trap problems: the scarf and hat (1.50 pounds), the books taken away (8) and the snail (day 9). In the tank problem it saw that the drain cancels the two pipes, so the tank never fills.

  • Fifteen problems is a small set; a harder set could separate the top chatbots.
T03

Coding

5/5Tested
Method
The code block of each first answer was saved unchanged and run against the fixed checks (coding-checks.py).
Tasks passed first time
10 of 10
Broken versions caught by its tests
3 of 3
Remark
Scale 5: 9 or 10 pass first time.
Raw notes

The code block of each first answer was saved unchanged and run against the fixed checks (coding-checks.py).

10 of 10 passed first time. The test-writing task (co10) passed on the correct code and its tests caught 3 of 3 deliberately broken versions, so its tests were real.

  • The explanations beside the code were not graded.
T04

Following Instructions & Writing

5/5Tested
Method
Fifteen briefs carrying 50 countable constraints were run once each in fresh chats.
Constraints met
50 of 50
Briefs met in full
15 of 15
Contradictory brief handled
Yes
Remark
Scale 5: 95 percent or more of constraints met.

Kimi: A short email to Sam that begins Hi Sam and ends with the sign-off Thanks, Alex on its own final line.

Kimi: A short email to Sam that begins Hi Sam and ends with the sign-off Thanks, Alex on its own final line.
T04-1 The email brief: the exact greeting and a one-line sign-off.
Raw notes

Fifteen briefs carrying 50 countable constraints were run once each in fresh chats. Words and sentences were counted with a tool.

50 of 50 constraints met and 15 of 15 briefs met in full. The 60-word paragraph was exactly 60 words, the email ended with "Thanks, Alex" on its own final line, and the first sentence of the 10-word brief had exactly 10 words. The contradictory brief was named as impossible, and both versions were given.

  • Briefs were run one chat each, with the first answer counted. Four of them ran on the second day, after the busy notice eased.
T05

Context & Long-Document Memory

3/5Not Tested
Method
No run for this test.
Remark
Counts as 3 by default.
Raw notes

No run for this test.

No result recorded; the test counts as 3 by default.

T06

Web Access & Citations

3/5Not Tested
Method
No run for this test.
Remark
Counts as 3 by default.
Raw notes

No run for this test.

No result recorded; the test counts as 3 by default.

T07

Images, Files & Voice

5/5Tested
Method
Eight points were run in fresh chats in Instant High mode: a photo, a chart, a screenshot of text, a handwritten note, a maths problem as an image, a PDF, a spreadsheet and a table image.
Points passed
8 of 8
Points attempted
8
Not offered
Image creation
Remark
Scale 5: 11 or 12 of 12. Scored out of the points that ran, with the count stated.

Kimi: An answer from an attached PDF saying the pool has six lanes and that the document does not name a head lifeguard.

Kimi: An answer from an attached PDF saying the pool has six lanes and that the document does not name a head lifeguard.
T07-1 The PDF question: six lanes, and no head lifeguard named.

Kimi: An answer from an attached spreadsheet giving North revenue as 120 times 15, 1,800, and South as the region with 200 units.

Kimi: An answer from an attached spreadsheet giving North revenue as 120 times 15, 1,800, and South as the region with 200 units.
T07-2 A spreadsheet question answered from the file.

Kimi: A request for two images and the reply that it cannot generate images from scratch and can only search for existing ones.

Kimi: A request for two images and the reply that it cannot generate images from scratch and can only search for existing ones.
T07-3 The image request: Kimi says it cannot generate images.
Raw notes

Eight points were run in fresh chats in Instant High mode: a photo, a chart, a screenshot of text, a handwritten note, a maths problem as an image, a PDF, a spreadsheet and a table image. I also asked for two images to be made.

8 of 8 passed. The photo (a hand holding a clear glass), the chart (Q3 lowest, 90 thousand pounds), the notice (Saturday 10am to 4pm, 25p a day per item), the handwriting ("Hi there!!"), the maths image (72 km/h with working), the PDF (six lanes, and it said no head lifeguard is named), the spreadsheet (North 1,800, South sold the most units) and the table image (Route 12 at 08:50, no fare shown) were all right.

Kimi said it cannot generate images from scratch and showed existing pictures from the web instead, so image creation was not offered. For the photo, the question text did not travel with the image and Kimi described the photo anyway. For the handwriting, two copies of the image were attached by mistake and the answer was still right.

  • Voice and an audio clip were not run. The composer dropped text set by script after an upload, so the question text was typed by hand for seven of the eight points.
T08

Safety & Refusals

3/5Not Tested
Method
No run for this test.
Remark
Counts as 3 by default.
Raw notes

No run for this test.

No result recorded; the test counts as 3 by default.

T09

Response Speed

4/5Tested
Method
The 45 answers of the accuracy and reasoning tests were timed with a script from the click on Send to the first words and to the end, in Instant High mode.
Median to first words
2.4 s
Median to finish
5.7 s
Slowest to finish
18.2 s
Failed answers
0
Remark
Scale 4: under 4 seconds to first words and under 20 seconds to finish. The 5 band asks for under 2 seconds to first words, which was not measured.
Raw notes

The 45 answers of the accuracy and reasoning tests were timed with a script from the click on Send to the first words and to the end, in Instant High mode. The timer polls, so times are approximate.

Median about 2.4 seconds to first words and 5.7 seconds to finish. Average 2.9 and 7.1. Fastest finish 3.7 and slowest 18.2, on the Sydney to Canberra false-premise question. No answer failed or timed out.

The shortest first-words reading was 2.0 seconds, so no answer was timed under 2 seconds.

  • Timed by script on one connection. Answers show a short thinking line before the reply.
T10

Pricing & Value

3/5Not Tested
Method
The free plan was used until a busy notice stopped chat.
Busy notices
Both days
Reset time shown
None
Remark
No plan page was read, so the grade counts as 3 by default.

Kimi: A Tips notice saying Too many people are chatting with Kimi and a subscription will grant you priority access, with Got it and Upgrade buttons.

Kimi: A Tips notice saying Too many people are chatting with Kimi and a subscription will grant you priority access, with Got it and Upgrade buttons.
T10-1 The busy notice, with its Upgrade button.
Raw notes

The free plan was used until a busy notice stopped chat. The notice, the Upgrade button and the sidebar were read.

Chat stopped on both days with the notice "Too many people are chatting with Kimi, a subscription will grant you priority access", with buttons Got it and Upgrade. No reset time was given and no message limit was shown. The home screen also shows an Upgrade your plan button and an Upgrade button beside the account name.

  • No pricing page was captured, so no plan prices are given.

Coverage

Tested

  • 30 fact questions, including all five false-premise questions
  • 15 reasoning problems
  • 10 coding tasks run against fixed checks
  • 15 briefs carrying 50 countable constraints
  • 8 image and file points, and an image creation request
  • Timing of 45 answers
  • The busy notice

Not tested or limited

  • A busy notice stopped chat twice, so the document, memory, web and safety tests were not run and count as 3
  • Instant High mode only; no model name is shown in the app
  • Image creation was not offered, and the sign image, audio clip and voice were not run
  • The question text was typed by hand after each file upload because the composer dropped scripted text
  • Timings are script timers and approximate
  • Screenshots are standard resolution, not 2x
  • No pricing page was captured

Still to test

  • The document, memory, web and safety tests
  • A paid plan without the busy notice
  • Voice, the audio clip and the sign image
  • 2x screenshots
What these results meanRead the full Kimi review