Chatbot coding checks
Code counts only when it runs and passes, so a good explanation cannot hide a bug.
Checker · AI chatbot
- When it runs
- After the coding test of every chatbot review
- Used in
- 10 reviews
- Runs on
- A Python script (coding-checks.py) on my PC
What the chatbot coding checks is
The ten tasks are four small functions with edge cases, two bug fixes, two data tasks, one script and one task that asks for tests. The checker runs each saved file against its cases.
The task that asks the chatbot to write tests is run against three deliberately broken versions, and its tests must fail each one.
How it works
- Save the code block of each first answer unchanged.
- Run the checker over the folder.
- Count the tasks that pass.
What it records
- Per task
- Pass or fail, and the case that failed.
What it cannot catch
- Only the code is graded here; how well it is explained is noted separately.
Where it was used
Every published review that ran this checker, with the figure it produced there, read from the review’s own data. Each figure opens the test results behind it.
| Review | Figure | Tested |
|---|---|---|
| 9 of 10 tasks passed | Invalid Date | |
| 10 of 10 tasks passed | Invalid Date | |
| 10 of 10 tasks passed | Invalid Date | |
| 10 of 10 tasks passed | Invalid Date | |
| 10 of 10 tasks passed | Invalid Date | |
| 4 of 4 tasks passed | Invalid Date | |
| 9 of 10 tasks passed | Invalid Date | |
| 10 of 10 tasks passed | Invalid Date | |
| 10 of 10 tasks passed | Invalid Date | |
| 8 of 8 tasks passed | Invalid Date |
Review
- Figure
- 9 of 10 tasks passed
- Tested
- Invalid Date
- Figure
- 10 of 10 tasks passed
- Tested
- Invalid Date
- Figure
- 10 of 10 tasks passed
- Tested
- Invalid Date
- Figure
- 10 of 10 tasks passed
- Tested
- Invalid Date
- Figure
- 10 of 10 tasks passed
- Tested
- Invalid Date
- Figure
- 4 of 4 tasks passed
- Tested
- Invalid Date
- Figure
- 9 of 10 tasks passed
- Tested
- Invalid Date
- Figure
- 10 of 10 tasks passed
- Tested
- Invalid Date
- Figure
- 10 of 10 tasks passed
- Tested
- Invalid Date
- Figure
- 8 of 8 tasks passed
- Tested
- Invalid Date
The other checkers for ai chatbot
Written by Miles Foster, Owner and tester, RankDude. How RankDude tests and grades is on the methodology page.