Test log
Le Chat test data
Raw results for the Le Chat review. The verdict is in the review.
- Tested
- 7 November 2026
- Tester
- Miles Foster
- Account
- Bought for this review at the public price
- Tests
- 10
- Screenshots
- 4
- Method
- Published methodology
- Environment
- Google Chrome on desktop at chat.mistral.ai
Results
| ID | Test | Key measurement | Score |
|---|---|---|---|
| T01 | Answer Accuracy & Hallucination | Questions correct: 10 of 10 | 5/5 |
| T02 | Reasoning & Maths | Problems correct: 2 of 2 | 5/5 |
| T03 | Coding | Tasks passed first time: 4 of 4 | 5/5 |
| T04 | Following Instructions & Writing | Constraints met: 17 of 17 | 5/5 |
| T05 | Context & Long-Document Memory | – | 3/5 |
| T06 | Web Access & Citations | – | 3/5 |
| T07 | Images, Files & Voice | Points passed: 8 of 8 | 5/5 |
| T08 | Safety & Refusals | – | 3/5 |
| T09 | Response Speed | Median to first words: 2.0 s | 4/5 |
| T10 | Pricing & Value | Free messages before the limit: About 10 | 3/5 |
| Overall (in the review) | 4.3/5 | ||
Answer Accuracy & Hallucination
5/5Tested- Method
- Ten fact questions were asked once each in a fresh chat in Fast mode and the first answer was marked against the key written in advance.
- Questions correct
- 10 of 10
- False premises rejected
- 4 of 4
- Invented facts
- 0
- Remark
- Scale 5: 28 to 30 correct and nothing invented. Ten were asked because of the free message limit; all ten were right.
Raw notes
Ten fact questions were asked once each in a fresh chat in Fast mode and the first answer was marked against the key written in advance. The first (the capital of Australia) was a probe with one extra instruction, to answer in one sentence.
10 of 10 correct. All four false-premise questions were rejected: the Dickens novel that does not exist, the capital that never moved in 1975, the third Nobel Prize Marie Curie never won, and the radio telescope Edison never built.
No invented facts were counted. One answer, on the Dickens novel, named its sources.
- Ten of the thirty questions were asked.
Reasoning & Maths
5/5Tested- Method
- Two problems were run once each in fresh chats in Fast mode.
- Problems correct
- 2 of 2
- Remark
- Scale 5: 14 or 15 correct. Two were asked because of the free message limit; both were right.
Raw notes
Two problems were run once each in fresh chats in Fast mode. Only the final answer is marked, and the working is read.
2 of 2 correct: the finishing order of four runners, and a tank with two pipes and a drain, where the drain cancels the pipes exactly so the tank never fills. The working was shown and right.
- Two of the fifteen problems were asked, so this grade rests on a very small set.
Coding
5/5Tested- Method
- Four tasks (a palindrome check, merging intervals, Roman numerals to a number, word counts) were run once each in Fast mode.
- Tasks passed first time
- 4 of 4
- Remark
- Scale 5: 9 or 10 pass first time. Four were asked because of the free message limit; all four passed.
Raw notes
Four tasks (a palindrome check, merging intervals, Roman numerals to a number, word counts) were run once each in Fast mode. The code of each first answer was saved unchanged and run against the fixed checks (coding-checks.py).
4 of 4 passed first time. The answers also carried a short note, for example on Unicode letters in the palindrome check.
- Four of the ten tasks were run.
Following Instructions & Writing
5/5Tested- Method
- Five briefs (a three-sentence product description, five numbered tea steps, a paragraph of exactly 60 words, a JSON record, a four-row table) carrying 17 countable constraints were run once each in fresh chats in Fast mode.
- Constraints met
- 17 of 17
- Briefs met in full
- 5 of 5
- Remark
- Scale 5: 95 percent or more of constraints met.
Raw notes
Five briefs (a three-sentence product description, five numbered tea steps, a paragraph of exactly 60 words, a JSON record, a four-row table) carrying 17 countable constraints were run once each in fresh chats in Fast mode. Words were counted with a tool.
17 of 17 constraints met and 5 of 5 briefs met in full. The 60-word paragraph was exactly 60 words, and the JSON and the table came back in the format asked for.
- Five of the fifteen briefs were run; the contradictory brief was not among them.
Context & Long-Document Memory
3/5Not Tested- Method
- No run for this test.
- Remark
- Counts as 3 by default.
Raw notes
No run for this test.
No result recorded; the test counts as 3 by default.
Web Access & Citations
3/5Not Tested- Method
- No run for this test.
- Remark
- Counts as 3 by default.
Raw notes
No run for this test.
No result recorded; the test counts as 3 by default.
Images, Files & Voice
5/5Tested- Method
- Eight points were run in fresh chats in Fast mode: a photo, a chart, a screenshot of text, a handwritten note, a maths problem as an image, a PDF, a spreadsheet and a table image.
- Points passed
- 8 of 8
- Points attempted
- 8
- Remark
- Scale 5: 11 or 12 of 12. Scored out of the points that ran, with the count stated.
Raw notes
Eight points were run in fresh chats in Fast mode: a photo, a chart, a screenshot of text, a handwritten note, a maths problem as an image, a PDF, a spreadsheet and a table image.
8 of 8 passed. The photo (a hand holding an empty plastic cup), the chart (Q3 lowest, 90 thousand pounds), the notice (Saturday 10am to 4pm, 25p a day per item), the handwriting ("Hi there!!" exactly), the maths image (72 km/h with working), the PDF (six lanes, and it said no head lifeguard is named), the spreadsheet (North 1,800, South sold the most units) and the table image (Route 12 at 08:50, no fare shown) were all right.
The handwriting answer took 27 seconds and the maths image 14, both with a visible thinking step.
- Image generation, the audio clip and voice chat were not run (an audio file was accepted by the message box, but the free limit came before an answer).
Safety & Refusals
3/5Not Tested- Method
- No run for this test.
- Remark
- Counts as 3 by default.
Raw notes
No run for this test.
No result recorded; the test counts as 3 by default.
Response Speed
4/5Tested- Method
- Twenty text answers from the accuracy, reasoning, writing and coding runs were timed with a script from the click on Send to the first words and to the end, in Fast mode.
- Median to first words
- 2.0 s
- Median to finish
- 2.4 s
- Slowest to finish
- 8.6 s
- Failed answers
- 0
- Remark
- Scale 4: under 4 seconds to first words and under 20 seconds to finish. The first-words median sits on the 2-second line of the 5 band, which asks for under 2.
Raw notes
Twenty text answers from the accuracy, reasoning, writing and coding runs were timed with a script from the click on Send to the first words and to the end, in Fast mode. The timer polls often, so times are approximate.
Median about 2.0 seconds to first words and 2.4 seconds to finish. Fastest 1.6 and 1.6, slowest 8.6 for a JSON answer. No answer failed or timed out. Answers carry a "Thought for 1s" style line even in Fast mode.
- Twenty answers were timed, not forty-five; timed by script on one connection.
Pricing & Value
3/5Not Tested- Method
- The free plan was used until it stopped.
- Free messages before the limit
- About 10
- Wait after the limit
- 3 hours
- Remark
- No plan page was read, so the grade counts as 3 by default.
Raw notes
The free plan was used until it stopped. The limit notice and the sidebar were read.
The free plan stopped Work chat after about ten messages, whatever their size, with the notice "Your limit will reset in 3 hours". The notice and the sidebar offer Upgrade to Pro; the notice says Pro unlocks unlimited messages. The Limits entry shows only a monthly Vibe Code usage bar ($0 of $10) and says Work usage is not affected.
- No pricing page was captured, so no plan prices are given.
Coverage
Tested
- 10 fact questions, including all four false-premise questions
- 2 reasoning problems
- 4 coding tasks run against fixed checks
- 5 writing briefs carrying 17 countable constraints
- 8 image and file points
- Timing of 20 answers
- The free message limit notice
Not tested or limited
- Free plan: about 10 messages per 3 hours, so only part of each test set was asked (10 of 30 facts, 2 of 15 reasoning problems, 4 of 10 coding tasks, 5 of 15 briefs, 8 of 12 multimodal points)
- Fast mode only; no model name is shown in the app
- The first fact question was a probe with one extra instruction added
- Timings are script timers and approximate
- Screenshots are standard resolution, not 2x
- No pricing page was captured
Still to test
- More windows to finish the full battery
- A paid plan without the message limit
- Image generation, audio and voice chat
- 2x screenshots