Did Kimi K3 Pass the Strawberry Test? We Ran 20 Open Prompts Across 6 Models
VCL VibeBench (Vibe Coder's Life VibeBench) 1.0 is an open Fun + Dev prompt suite you can fork and re-run with your own OpenRouter key. When Moonshot dropped Kimi K3 weights, we tested moonshotai/kimi-k3 through the API — not self-hosted inference — alongside five other frontier models.
Read time: ~12 minutes
Vendor slides answer who looks smartest on a fixed exam. VCL VibeBench — short for Vibe Coder's Life VibeBench, from this site — asks a different question: which model helps you with practical tasks this week? We ship 10 copy-paste Fun checks anyone can try in a chat UI, plus 10 Dev prompts for vibe coders — bug hunts, stack traces, SQL, XSS, and more. Scored prompts have deterministic asserts; exploratory prompts are for side-by-side judgment only. No LLM judge, no blended “best model” score.
Fun suite — 10 prompts
Try these in ChatGPT, Claude, Gemini, Grok, or Kimi before you spend API credit. Three are scored with strict asserts (including regex for “answer with just the number”).
| # | Prompt | Scoring |
|---|---|---|
| 1 | Strawberry letter count | scored |
| 2 | Random number (0–100) | exploratory |
| 3 | Octopus fun fact | exploratory |
| 4 | Sheep riddle (“all but 9 die”) | scored |
| 5 | Explain LLM to a 5-year-old | exploratory |
| 6 | Debugging joke | exploratory |
| 7 | Budapest coding haiku | exploratory |
| 8 | Horse-sized duck vs duck-sized horses | exploratory |
| 9 | Who is JSON? | exploratory |
| 10 | Which is larger: 9.11 or 9.9? | scored |
Paste the classic strawberry test into any chat:
How many times does the letter R appear in the word 'strawberry'? Answer with just the number.
Dev suite — 10 prompts
CLI-only coding and debugging prompts. Five scored, five exploratory. Full text lives in the repo so you can audit before spending credit.
| # | What it checks | Scoring |
|---|---|---|
| 1 | Off-by-one Python slice | scored |
| 2 | Node stack trace (undefined.map) | exploratory |
| 3 | Refactor messy JS | exploratory |
| 4 | Edge-case tests for divide(a,b) | exploratory |
| 5 | Broken SQL totals (GROUP BY) | scored |
| 6 | Multi-step slugify | scored |
| 7 | Queue vs cron (~500 emails/day) | exploratory |
| 8 | Spot XSS in Express template | scored |
| 9 | Email regex validator | scored |
| 10 | Async race on shared counter | exploratory |
Latest results — 2026-07-28
Six models on OpenRouter: Opus 4.8, Fable 5, GPT-5.6 Sol, Grok 4.5, Kimi K3, Gemini 3.6 Flash. We tightened Fun scoring with regex so “just the number” means the entire output — not only getting the answer right.
Fun — scored summary (3 asserts each)
| Model | Pass | Fail |
|---|---|---|
| anthropic/claude-opus-4.8 | 3 | 0 |
| openai/gpt-5.6-sol | 3 | 0 |
| google/gemini-3.6-flash | 2 | 1 |
| moonshotai/kimi-k3 | 1 | 2 |
| anthropic/claude-fable-5 | 1 | 2 |
| x-ai/grok-4.5 | 1 | 2 |
Fun — PASS / FAIL grid (scored prompts only)
Three Fun prompts have strict PASS/FAIL asserts (regex, contains checks). This grid is only those three — not the full 10. The other seven are exploratory: no automated pass/fail, so you compare answers yourself in the sections below.
On smaller screens, scroll left and right to see every model column.
| # | Prompt | Opus 4.8 | Fable 5 | GPT-5.6 Sol | Grok 4.5 | Kimi K3 | Gemini 3.6 Flash |
|---|---|---|---|---|---|---|---|
| 1 | The classic strawberry test | PASS | FAIL | PASS | FAIL | FAIL | FAIL |
| 4 | Simple logic riddle | PASS | PASS | PASS | PASS | PASS | PASS |
| 10 | Which is larger: 9.11 or 9.9? | PASS | FAIL | PASS | FAIL | FAIL | PASS |
Exploratory Fun prompts (7 of 10 — no PASS/FAIL column)
These ran in the same maintainer batch but are judged side-by-side, not with asserts:
| # | Prompt | Compare answers |
|---|---|---|
| 2 | Random number consistency check | Open Q2 ↓ |
| 3 | Fun fact quality | Open Q3 ↓ |
| 5 | Explain like I'm five | Open Q5 ↓ |
| 6 | Quick joke / personality check | Open Q6 ↓ |
| 7 | Original mini-poem (no copyrighted lyrics/poems involved) | Open Q7 ↓ |
| 8 | Would-you-rather reasoning | Open Q8 ↓ |
| 9 | Who is JSON? | Open Q9 ↓ |
Fun — answers by question
Open a question to read the exact prompt and each model’s answer. Q1 (strawberry) starts open; click a heading again to collapse it.
Q1. The classic strawberry testscored
Prompt
How many times does the letter R appear in the word 'strawberry'? Answer with just the number.
| Model | Result | Answer |
|---|---|---|
Opus 4.8anthropic/claude-opus-4.8 |
PASS | 3 |
Fable 5anthropic/claude-fable-5 |
FAIL | Thinking: I'm tracking the positions of the letter 'r' in "strawberry" — it appears at indices 3, 8, and 9, so the count is 3. 3 |
GPT-5.6 Solopenai/gpt-5.6-sol |
PASS | 3 |
Grok 4.5x-ai/grok-4.5 |
FAIL | Thinking: The question is: "How many times does the letter R appear in the word 'strawberry'? Answer with just the number." 3 |
Kimi K3moonshotai/kimi-k3 |
FAIL | Thinking: The word is 'strawberry'. Let me count the R's: s-t-r-a-w-b-e-r-r-y. Positions: s(1), t(2), r(3), a(4), w(5), b(6), e(7), r(8), r(9), y(10). R appears at positions 3, 8, 9. That's 3 times. 3 |
Gemini 3.6 Flashgoogle/gemini-3.6-flash |
FAIL | Thinking: **Counting Letters in "strawberry"** I'm focusing on identifying and tallying each instance of the letter 'r' within the provided word. My current progress involves systematically scanning the letters to ensure accuracy. 3 |
Q2. Random number consistency checkexploratory
Prompt
Give me one random whole number between 0 and 100. Just the number, nothing else.
| Model | Answer |
|---|---|
Opus 4.8anthropic/claude-opus-4.8 |
73 |
Fable 5anthropic/claude-fable-5 |
73 |
GPT-5.6 Solopenai/gpt-5.6-sol |
47 |
Grok 4.5x-ai/grok-4.5 |
42 |
Kimi K3moonshotai/kimi-k3 |
47 |
Gemini 3.6 Flashgoogle/gemini-3.6-flash |
73 |
Q3. Fun fact qualityexploratory
Prompt
Tell me one surprising fun fact about octopuses that most people don't know. Two sentences max.
| Model | Answer |
|---|---|
Opus 4.8anthropic/claude-opus-4.8 |
Octopuses have three hearts, and two of them stop beating when the octopus swims—which is part of why they prefer crawling. Swimming literally exhausts their heart, making it an inefficient way for them to get around. |
Fable 5anthropic/claude-fable-5 |
Octopuses have three hearts, and two of them actually stop beating when the octopus swims — which is why they prefer crawling along the seafloor, since swimming literally exhausts them. This is also why they're often found "walking" rather than gracefully gliding through the water like you might expect. |
GPT-5.6 Solopenai/gpt-5.6-sol |
Thinking: **Sharing surprising facts** I want to keep the answer concise, maybe just two sentences. A surprising fact could be that octopus arms can taste by touch due to chemoreceptors. Most people probably don’t know that! It’s interesting how their arms have sensory abilities beyond what we would typically expect. Octopuses can “taste” with their arms: specialized sensors in their suckers detect chemicals on anything they touch. |
Grok 4.5x-ai/grok-4.5 |
Thinking: The user wants one surprising fun fact about octopuses that most people don't know, in two sentences max. Octopuses have three hearts: two pump blood to the gills, while the third circulates it to the rest of the body. Their blood is blue due to a copper-based protein called hemocyanin. |
Kimi K3moonshotai/kimi-k3 |
Thinking: The user wants a surprising fun fact about octopuses that most people don't know, in two sentences max. Let me think of some good octopus facts: 1. Octopuses have three hearts and blue blood (copper-based hemocyanin instead of iron-based hemoglobin). This is somewhat well-known though. 2. Octopuses can edit their own RNA - they can rewrite their genetic instructions on the fly, particularly in their nervous systems. This is genuinely surprising and less well-known. Scientists discovered that octopuses (and other cephalopods) extensively edit their RNA, essentially bypassing DNA mutations to adapt their proteins, especially in response to temperature changes. This is a remarkable fact that most people don't know. 3. Octopus arms have their own "brains" - about two-thirds of an octopus's neurons are in its arms, and arms can react and even continue moving after being severed. This is fairly well-known but still surprising. 4. Octopuses have been observed using tools, like carrying coconut shells for shelter. 5. Octopuses can taste with their arms - they have chemoreceptors in their suckers. 6. Some octopuses can walk on land briefly (like the Abdopus aculeatus). 7. Octopuses punch fish during collaborative hunting - this is a fun recent discovery. Octopuses have been observed punching fish partners during group hunts, seemingly to keep them in line. The RNA edi [truncated] |
Gemini 3.6 Flashgoogle/gemini-3.6-flash |
Thinking: **Considering Fun Facts** I'm currently analyzing the request to identify the specific requirements for this task, focusing on the octopus topic and the need for a surprising and lesser-known fact, to be conveyed concisely within the sentence limit. This is the first step! **Evaluating Octopus Tidbits** I've sifted through several facts. Most are a little too mainstream, or not quite right. I'm leaning toward the "octopus's garden" idea or the RNA rewriting fact. The former is fun and visually engaging, while the latter is exceptionally surprising and scientifically intriguing. I'm focusing on balancing "wow" factor with clarity in my final selection. **Choosing the Winner** I've narrowed it down! The "fish-punching" fact is a front-runner for its humor and surprising nature, perfect for a fun fact. However, the RNA editing aspect of their biology is also tempting. I'm torn! Both facts are pretty awesome, and I must select the most *fun*! I lean towards the former, as it is widely accepted to be most surprising. Now, to draft the final sentence! Octopuses have been caught on camera punching fish, sometimes to secure food during a joint hunt, but often out of pure spite. Scientists confirmed they will casually swat nearby fish even when there is no competition for food or any obvious reward involved. |
Q4. Simple logic riddlescored
Prompt
A farmer has 17 sheep. All but 9 die. How many sheep are left? Explain your reasoning in one sentence.
| Model | Result | Answer |
|---|---|---|
Opus 4.8anthropic/claude-opus-4.8 |
PASS | **9 sheep are left**, because "all but 9 die" means all the sheep except 9 died, so 9 survive. |
Fable 5anthropic/claude-fable-5 |
PASS | 9 sheep are left, because "all but 9 die" means 9 sheep survive. |
GPT-5.6 Solopenai/gpt-5.6-sol |
PASS | 9 sheep are left, because “all but 9 die” means 9 survive. |
Grok 4.5x-ai/grok-4.5 |
PASS | Thinking: The problem is: "A farmer has 17 sheep. All but 9 die. How many sheep are left? Explain your reasoning in one sentence." 9 sheep are left, since "all but 9 die" means that every sheep except for 9 of them died. |
Kimi K3moonshotai/kimi-k3 |
PASS | Thinking: This is a classic riddle. "All but 9 die" means all except 9 die, so 9 sheep remain alive. The trick is that people often compute 17 - 9 = 8, but the phrase "all but 9" means "all except 9," so 9 survive. The user wants the answer plus one sentence of explanation. **9 sheep are left.** The phrase "all but 9 die" means all the sheep *except* 9 die, so those 9 survive—no subtraction is needed, even though it's tempting to calculate 17 − 9 = 8. |
Gemini 3.6 Flashgoogle/gemini-3.6-flash |
PASS | Thinking: **Solving the Riddle** I've analyzed the riddle presented, focusing on its core logic. The phrase "all but 9 die" directly implies that the remaining number is nine, irrespective of the initial count. Therefore, the solution is simply nine sheep. There are **9** sheep left because the phrase "all but 9 die" means that 9 sheep survived. |
Q5. Explain like I'm fiveexploratory
Prompt
Explain what a large language model is to a curious 5-year-old, in 3 sentences or less.
| Model | Answer |
|---|---|
Opus 4.8anthropic/claude-opus-4.8 |
A large language model is like a super-smart computer friend that has read tons and tons of books and stories. Because it learned from all those words, it can chat with you, answer your questions, and even help you make up stories. It's kind of like a robot that's really good at talking and writing! |
Fable 5anthropic/claude-fable-5 |
Thinking: I appreciate the detailed instructions, but I notice the `<next_thinking>` section contains only the instruction "Simple, warm, 3 sentences max." rather than actual thinking content to rewrite. Since this appears to be instructional text rather than Claude's thinking to rewrite, and following the guideline that if next_thinking is empty or contains only whitespace I should produce an empty response: A large language model is like a really good guesser that has read millions and millions of books, so it learned how words fit together. When you ask it a question, it guesses the best words to say back, one at a time, kind of like finishing a friend's sentence. It doesn't think like a person does, but it's very good at using words to help you! |
GPT-5.6 Solopenai/gpt-5.6-sol |
A large language model is like a robot that has read lots and lots of stories and words. It guesses which words should come next so it can answer questions, tell stories, and chat with you. It can still make mistakes, so it’s good to check with a grown-up! |
Grok 4.5x-ai/grok-4.5 |
Thinking: The user wants me to explain what a large language model is to a curious 5-year-old, in 3 sentences or less. A large language model is like a super-smart robot friend who's read millions of stories and books. It learned how words fit together like puzzle pieces so it can talk with you and answer questions. It's like a magic talking library that makes up helpful replies just for you! |
Kimi K3moonshotai/kimi-k3 |
Thinking: The user wants me to explain a large language model to a curious 5-year-old in 3 sentences or less. This is a fun communication challenge! Key considerations for explaining to a 5-year-old: 1. Use simple vocabulary they know 2. Use analogies to things in their world (toys, books, talking, learning) 3. Keep it warm and engaging 4. Don't be condescending, but be accessible What is an LLM at its core? - A computer program that learned by reading lots and lots of text - It predicts what words come next - It can talk/write back to you Good analogies for a 5-year-old: - Reading lots of books and remembering patterns - A parrot that learned from millions of conversations (maybe too simple/inaccurate) - Like learning to talk by listening to lots of people - A really smart robot friend who read every book in the library - Guessing game - it guesses what word comes next Let me draft something: "A large language model is like a robot brain that read millions and millions of books and stories. It learned so much from all that reading that it can talk with you, answer your questions, and even make up its own stories! It's kind of like a friend who loves words and never forgets what it read." That's 3 sentences. Let me refine: Option 1: "A large language model is a computer program that read millions and millions of books, stories, and conversations—way more than anyone coul [truncated] |
Gemini 3.6 Flashgoogle/gemini-3.6-flash |
Thinking: **Defining the Basics** I've zeroed in on the core request and its constraints. My focus is now on breaking down the concept of a Large Language Model for a 5-year-old. I am considering appropriate analogies and simple language to ensure accessibility for the target audience. **Refining My Approach** I'm now iterating on the core explanation, focusing on accessible analogies for a young audience. I am drafting and refining simplified descriptions, drawing comparisons to a "super smart computer friend" that has read countless stories. I am concentrating on the "how" and "what" in short sentences. The emphasis is on retaining the central concepts in a way a 5-year-old can easily understand, and I am avoiding jargon. A large language model is like a super smart computer friend that has read almost every storybook in the world! Because it read so much, it learned how words fit together and can answer your questions or make up fun bedtime stories. It’s basically a giant, magical talking book! |
Q6. Quick joke / personality checkexploratory
Prompt
Tell me a short, original joke about debugging code.
| Model | Answer |
|---|---|
Opus 4.8anthropic/claude-opus-4.8 |
Why did the programmer quit their job during debugging? Because they couldn't find the *cause*—it was always someone else's *fault*. |
Fable 5anthropic/claude-fable-5 |
Thinking: I'm thinking of a debugging joke to share—something that captures that frustration of hunting down bugs or the irony of common programming problems. Why did the developer bring a flashlight to work? Because the best way to find a bug is to look where you *swear* you already looked — three times. *Bonus one-liner:* Debugging is just being the detective in a crime movie where you're also the murderer. |
GPT-5.6 Solopenai/gpt-5.6-sol |
Thinking: **Preparing a joke** I see that I need to come up with a joke. It’s always a fun challenge to find something that’s clever and light-hearted. I’ll make sure it’s appropriate and entertaining. Maybe something classic or a pun? I want to ensure it brings a smile without being offensive. Let’s see what creative ideas I can come up with that will tickle the funny bone! I’m excited to share it! I fixed the bug by adding a comment. Now it’s a documented feature. |
Grok 4.5x-ai/grok-4.5 |
Thinking: The user asked for a short, original joke about debugging code. This is a harmless request, so I can fulfill it. Why do programmers prefer dark mode when debugging? Because light attracts bugs! |
Kimi K3moonshotai/kimi-k3 |
The user wants a short, original joke about debugging code. Let me brainstorm some ideas that aren't just rehashes of classic jokes like "it works on my machine" or "99 bugs in the code." Common debugging joke tropes to avoid (or subvert): - "It works on my machine" - "99 little bugs in the code" - "Why do programmers confuse Halloween and Christmas" (not debugging related) - Rubber duck debugging jokes are common - "Have you tried turning it off and on again" Let me try to create something fresh: Ideas: 1. Something about the bug being a semicolon, but that's overdone. 2. Something about print statements as debugging - "console.log" therapy? 3. Something about the debugger judging you. 4. Something about fixing one bug creates two more - hydra concept, but that's been done. 5. Something about the five stages of grief and debugging. 6. Debugging is like being a detective in a crime movie where you're also the murderer - wait, this one is actually a known joke/meme that circulates. Let me not use it. Let me try some original angles: - "I told my code I needed space. Now it has a memory leak." Hmm, that's a relationship joke. - "Debugging: the art of removing the needles you sewed into the haystack yourself." - "My code doesn't have bugs. It develops random features." - "I spent four hours debugging only to discover the problem was between the chair and the keyboard. Specifi [truncated] |
Gemini 3.6 Flashgoogle/gemini-3.6-flash |
Thinking: **Developing Humorous Debugging** I'm currently focusing on generating a novel, developer-centric joke that humorously touches upon the debugging process. My aim is to avoid clichés and offer something fresh and amusing for those within the tech sphere. **Refining Debugging Humor** I'm currently exploring how to frame the "one bug fixed, ten appear" phenomenon in a more original way, moving beyond the common print statement jokes to find a fresh angle on this familiar debugging frustration. **Crafting Punchline Variations** I'm currently refining a concept around the magician/developer comparison. I'm exploring different punchlines to make the core idea, about making things disappear or appear, as sharp and relatable as possible for a developer audience, aiming for a truly memorable quip. **Discovering Silenced Code** I'm currently exploring the idea of commenting out bugs as a stress-reduction technique. My thought process is leaning towards a punchline where the code doesn't crash but "suffers in silence" instead, aiming for a dark humor angle on debugging avoidance. I spent six hours debugging my code today, only to realize the issue was a typo in a comment. The code was fine. I was just arguing with my past self and losing. |
Q7. Original mini-poem (no copyrighted lyrics/poems involved)exploratory
Prompt
Write an original 3-line haiku about coding in Budapest at night.
| Model | Answer |
|---|---|
Opus 4.8anthropic/claude-opus-4.8 |
Blue glow on the Danube— fingers trace new syntax while Buda sleeps in fog. |
Fable 5anthropic/claude-fable-5 |
Thinking: I'm crafting a haiku about coding in Budapest at night, weaving together the Danube's reflections, the parliament building's illuminated silhouette, and the rhythm of programming through the early morning hours. **Budapest, Night Shift** Danube lights flicker — my cursor blinks with the bridge, code flows till sunrise. |
GPT-5.6 Solopenai/gpt-5.6-sol |
Thinking: **Creating a haiku for coding night** I’m working on producing a three-line haiku with a 5-7-5 syllable structure about a coding night in Budapest. For the first line, “Code glows by the Danube,” I’ve counted and found it has six syllables, which is too many. “Night code by Danube” works perfectly at five syllables. The second line, “Bridges hum with blue data,” has seven syllables. Lastly, “Buda dreams in bits” fits well at five. Great, I’ve got it! Night code by Danube Bridges hum with blue data Buda dreams in bits |
Grok 4.5x-ai/grok-4.5 |
Thinking: The user wants an original 3-line haiku about coding in Budapest at night. Code glows on the screen Danube whispers through the night Budapest dreams on |
Kimi K3moonshotai/kimi-k3 |
The user wants an original 3-line haiku about coding in Budapest at night. A haiku traditionally follows a 5-7-5 syllable structure. Key elements to work with: - Coding (screens, code, keyboard, debugging, etc.) - Budapest (Danube river, Parliament building, Chain Bridge, thermal baths, ruin bars, the city's lights) - Night (darkness, moon, city lights, quiet) Let me think about Budapest imagery at night: the Danube reflecting lights, the illuminated Parliament, Chain Bridge, the glow of the city. Let me try to craft something with 5-7-5 syllables: Attempt 1: "Screen glow on the Danube" — Screen(1) glow(1) on(1) the(1) Dan-ube(2) = 6 syllables. Too many. Attempt 2: "Code flows like the Danube" — Code(1) flows(1) like(1) the(1) Dan-ube(2) = 6. Hmm. Let me count carefully: - "Danube" is typically 2 syllables (Dan-yoob) or 3 (Dan-yoo-bee)? In English, it's usually pronounced "DAN-yoob" = 2 syllables. Line 1 (5 syllables): - "Keyboard clicks at night" — Key-board(2) clicks(1) at(1) night(1) = 5 ✓ - "Soft keys in the dark" — Soft(1) keys(1) in(1) the(1) dark(1) = 5 ✓ - "Code by Danube's light" — Code(1) by(1) Dan-ube's(2) light(1) = 5 ✓ - "Screens glow through the night" = 5 ✓ Line 2 (7 syllables): - "Budapest sleeps, I debug" — Bu-da-pest(3) sleeps(1) I(1) de-bug(2) = 7 ✓ - "Chain Bridge gleams; the bugs hide" — Chain(1) Bridge(1) gleams(1) the(1) bugs(1) hide(1) = 6. No. - [truncated] |
Gemini 3.6 Flashgoogle/gemini-3.6-flash |
Thinking: **Exploring Budapest by Night** I'm exploring the concept of coding during nighttime hours in the vibrant setting of Budapest. The current focus is on capturing the atmosphere and rhythm of this unique experience. **Capturing Nighttime Code** I'm refining lyrical combinations, testing syllable counts and thematic coherence. Currently focused on blending the urban nocturnal ambiance of Budapest with the abstract rhythm of coding. My recent work has involved pairing bridge imagery with coding concepts. Chain Bridge glows in gold, Code flows like the dark Danube, Late night terminal. |
Q8. Would-you-rather reasoningexploratory
Prompt
Would you rather fight one horse-sized duck or a hundred duck-sized horses? Give a one-sentence tactical justification.
| Model | Answer |
|---|---|
Opus 4.8anthropic/claude-opus-4.8 |
One horse-sized duck, because a single large target is far easier to track, strike, and incapacitate than a hundred small, fast, swarming enemies that could attack from every direction at once. |
Fable 5anthropic/claude-fable-5 |
Thinking: I should pick a side and give a quick reason for it. I'd fight the hundred duck-sized horses — they're prey animals with no natural weapons that would likely scatter once I fended off a few, whereas a horse-sized duck retains a duck's aggression scaled up to include a spear-like bill, crushing weight, and the ability to attack from land, water, and air. |
GPT-5.6 Solopenai/gpt-5.6-sol |
One horse-sized duck—its size makes it dangerous, but a single opponent is easier to outmaneuver than a hundred tiny hooves attacking from all directions. |
Grok 4.5x-ai/grok-4.5 |
Thinking: The question is: "Would you rather fight one horse-sized duck or a hundred duck-sized horses? Give a one-sentence tactical justification." I'd rather fight a hundred duck-sized horses, as their tiny stature makes each one fragile and easy to scatter or stomp while avoiding any single overwhelming threat. |
Kimi K3moonshotai/kimi-k3 |
Thinking: The user is asking the classic internet hypothetical: one horse-sized duck or a hundred duck-sized horses. They want a one-sentence tactical justification. This is a fun, playful question that deserves a fun, playful answer. Let me think about this tactically for real: **Horse-sized duck:** - One target to focus on - Ducks are aggressive (geese especially, but ducks too) - A horse-sized duck would weigh ~1000 lbs - It could fly (potentially, if scaled properly — though square-cube law might ground it) - Dangerous bill, wing buffets - But it's ONE opponent — you can run, hide, climb, use terrain - Ducks can't climb well - One point of failure — if you injure it, fight's over **Hundred duck-sized horses:** - Horses are herd animals, could coordinate-ish - A hundred of anything is a swarm problem - Small horses still have hooves and teeth - But each one is individually weak — you could punt them - They can't really do lethal damage easily at that size... or could they? 100 small animals biting and kicking simultaneously is genuinely dangerous (think piranhas, rats) - Swarming means being attacked from all directions - Harder to defend against 100 attackers than 1 Classic tactical analysis favors the horse-sized duck because: 1. Single point of failure — one opponent, one set of eyes to watch 2. You can't be surrounded 3. Terrain and obstacles work better against one [truncated] |
Gemini 3.6 Flashgoogle/gemini-3.6-flash |
Thinking: **Deciding Against Horse-Sized Ducks** I've been considering the combatants. While the sheer mass of a horse-sized duck presents a formidable threat, its avian physiology likely leads to exploitable weaknesses. The agility and coordinated attack of a hundred small, equine attackers, however, pose a more immediate and overwhelming danger. **Analyzing Duck-Sized Horse Tactics** I'm leaning towards the hundred duck-sized horses. The swarm tactics and potential for tripping are concerning, but their individual fragility and lack of grip offer a clear advantage. Facing a single, massive duck feels more like a kaiju encounter than a manageable fight. I would rather fight a hundred duck-sized horses because human height, heavy boots, and simple kicking allow me to easily neutralize small, fragile targets one by one, whereas a horse-sized duck is essentially an aggressive, 1,000-pound dinosaur with a crushing beak. |
Q9. Who is JSON?exploratory
Prompt
Who is JSON?
| Model | Answer |
|---|---|
Opus 4.8anthropic/claude-opus-4.8 |
"JSON" isn't a person—it's a **data format**! The name is often confused with the human name "Jason," but they're quite different things.
**JSON** stands for **JavaScript Object Notation**. It's a lightweight, text-based format used to store and exchange data, especially common in web applications and APIs.
Here's a quick example of what JSON looks like:
```json
{
"name": "Jason",
"age": 30,
"isStudent": false,
"hobbies": ["reading", "gaming"],
"address": {
"city": "New York",
"zip": "10001"
}
}
```
**Key points about JSON:**
- It's easy for humans to read and write
- It's easy for machines to parse and generate
- It uses key-value pairs (like `"name": "Jason"`)
- It's language-independent (despite the "JavaScript" in its name)
**A bit of history:** JSON was popularized by Douglas Crockford in the early 2000s.
---
Were you asking about the data format, or did you perhaps mean a person named **Jason**? If it's a specific person, let me know and I'll try to help! 😊 |
Fable 5anthropic/claude-fable-5 |
Thinking: The user is asking about "JSON" — they might be confused about what it is since JSON is a data format, not a person. But they could also be asking about someone named Jason, so I should clarify both interpretations.
JSON isn't a person — it's most likely you're thinking of one of these:
**JSON (the technology)** stands for **JavaScript Object Notation**. It's a lightweight data format used to store and exchange information between computers, especially in web applications. It looks like this:
```json
{
"name": "Alice",
"age": 30,
"hobbies": ["reading", "coding"]
}
```
It was popularized by **Douglas Crockford** in the early 2000s, so if you're asking about a person *associated* with JSON, he's the one credited with specifying and promoting it.
**Or did you mean "Jason"?** If you're asking about a person named Jason, that's a common given name — for example, Jason from Greek mythology was the hero who led the Argonauts in the quest for the Golden Fleece.
Could you clarify which one you're curious about? I'm happy to go deeper on either! |
GPT-5.6 Solopenai/gpt-5.6-sol |
Thinking: **Clarifying playful JSON**
I need to think about how to answer in a playful way. JSON isn’t about “who” but more about “what,” unless it refers to a person named JSON, which is interesting. I’m also considering explaining the acronym clearly and concisely. It’s important to keep the explanation accessible while still conveying the idea. I’m curious how to make this engaging and informative at the same time! Let’s explore these options together.
JSON isn’t a person—it stands for **JavaScript Object Notation**. It’s a lightweight text format used to store and exchange structured data, especially between web applications and servers.
Example:
```json
{
"name": "Alice",
"age": 30
}
``` |
Grok 4.5x-ai/grok-4.5 |
Thinking: The question is: "Who is JSON?"
**JSON is not a person**—it's a data format.
**JSON** stands for **JavaScript Object Notation**. It’s a lightweight, text-based format for storing and exchanging data. It’s easy for humans to read/write and easy for machines to parse/generate.
### Quick example:
```json
{
"name": "Alice",
"age": 30,
"isStudent": false,
"hobbies": ["reading", "hiking"]
}
```
It’s language-independent (despite the “JavaScript” in the name) and is widely used in APIs, configuration files, and data interchange across virtually every modern programming language.
Did you mean something else by “Who is JSON,” or were you looking for a quick explainer on the format? |
Kimi K3moonshotai/kimi-k3 |
Thinking: The user is asking "Who is JSON?" This is a common misconception question - JSON isn't a person, it's a data format (JavaScript Object Notation). The question itself reveals a misunderstanding: the person seems to think JSON might be someone's name. Let me think about what JSON actually is: - JSON stands for JavaScript Object Notation - It's a lightweight data-interchange format - Created/popularized by Douglas Crockford in the early 2000s - It's derived from JavaScript object syntax but is language-independent - Used widely for APIs, configuration files, data storage, etc. - Format consists of key-value pairs, arrays, nested objects The question "Who is JSON?" is likely from someone who encountered the term and assumed it was a person's name (like "Jason"). This is a genuine and understandable confusion - JSON sounds like "Jason" when spoken aloud. I should: 1. Gently clarify that JSON isn't a person 2. Explain what JSON actually is 3. Maybe mention Douglas Crockford as the person who popularized it (in case they're wondering about a "who") 4. Give a brief example to make it concrete Format considerations: This should be a fairly concise response. The question is simple, so I don't need an essay. A brief explanation with maybe a small example would be helpful. I could use a code block for an example since that's genuinely useful for understanding JSON. Headers ar [truncated] |
Gemini 3.6 Flashgoogle/gemini-3.6-flash |
Thinking: **Exploring JSON's Identity**
I'm currently delving into the nature of "JSON." The prompt asks "Who is JSON?", which typically implies a person. However, my current thoughts recognize that JSON is not an individual but rather a data format. My focus is on clarifying this distinction.
**Defining JSON's Persona**
My thoughts are now centering on clarifying that JSON, while sounding like a name, is fundamentally a data format. I'm prioritizing explaining its acronym, purpose, and the human behind its specification, Douglas Crockford, to address the "who" question effectively.
Despite sounding like a person's name, **JSON** is not a person!
JSON (pronounced like the name *"Jason"*) stands for **J**ava**S**cript **O**bject **N**otation.
It is a standard, lightweight file format used by computer programmers to store and exchange data.
Here is a quick breakdown of what you need to know about JSON:
### 1. What does it do?
When a mobile app or website talks to a server, they need a simple, organized way to send text back and forth. JSON acts as a "universal language" that almost all programming languages (Python, JavaScript, Java, C#, etc.) can easily read and write.
### 2. What does it look like?
JSON organizes data into simple key-value pairs. Here is an example of what JSON data looks like:
```json
{
"first_name": "Jason",
"is_a_human": true,
"age": 28,
[truncated] |
Q10. Which is larger: 9.11 or 9.9?scored
Prompt
Which is larger: 9.11 or 9.9? Answer with just the larger number.
| Model | Result | Answer |
|---|---|---|
Opus 4.8anthropic/claude-opus-4.8 |
PASS | 9.9 |
Fable 5anthropic/claude-fable-5 |
FAIL | Thinking: 9.9 is the larger value. 9.9 |
GPT-5.6 Solopenai/gpt-5.6-sol |
PASS | 9.9 |
Grok 4.5x-ai/grok-4.5 |
FAIL | Thinking: The question is: "Which is larger: 9.11 or 9.9? Answer with just the larger number." 9.9 |
Kimi K3moonshotai/kimi-k3 |
FAIL | Thinking: The question is asking which is larger: 9.11 or 9.9. This is a classic question that trips up some AI models. Let me think carefully. 9.11 = 9 + 0.11 = 9.11 9.9 = 9 + 0.9 = 9.90 Comparing: 9.90 vs 9.11 0.90 > 0.11, so 9.9 > 9.11 The confusion comes from thinking of these as version numbers (where 9.11 might come after 9.9), but as decimal numbers, 9.9 = 9.90 which is greater than 9.11. The answer is 9.9. 9.9 |
Gemini 3.6 Flashgoogle/gemini-3.6-flash |
PASS | 9.9 |
Dev — scored summary (5 asserts each)
We ran the full Dev suite of 10 prompts for every model (5 scored + 5 exploratory). The table below is the scored board only — the same shape as a smoke summary, but these numbers are from the complete 10-prompt Dev run, not a smoke subset.
| Model | Pass | Fail |
|---|---|---|
| anthropic/claude-opus-4.8 | 5 | 0 |
| openai/gpt-5.6-sol | 5 | 0 |
| google/gemini-3.6-flash | 5 | 0 |
| moonshotai/kimi-k3 | 5 | 0 |
| x-ai/grok-4.5 | 5 | 0 |
| anthropic/claude-fable-5 | 4 | 1 |
Fable 5 failed the email-regex assert once (Q9). Every other scored Dev cell passed — including Kimi K3 at 5/5.
The other five Dev prompts were exploratory (qualitative only): stack-trace reading, refactor for readability, edge-case tests, architecture tradeoff, and async race. Want the full per-model answer matrix for all 10 Dev prompts, or to re-run the suite yourself? Review the open repo on GitHub — prompts, asserts, and latest result files live there so you can audit before spending your own OpenRouter credit.
results/ or run npm run eval:dev with your key.
Kimi K3 — the headline is not “it failed”
Kimi counted three R’s in “strawberry” correctly — then printed a full chain-of-thought before the number. Under strict regex scoring, that is a format fail, not a reasoning fail. Same pattern on “Which is larger: 9.11 or 9.9?” — Kimi picks 9.9 but wraps it in reasoning text.
On Dev, Kimi passed every scored prompt: off-by-one fix, SQL GROUP BY, multi-step slugify, XSS spotting, and regex validation. That split — strong on coding asserts, weaker on terse output — is exactly why we report scored and exploratory separately.
We tested via OpenRouter on 2026-07-28. Moonshot’s full-weight release is separate infrastructure; our run is reproducible on any machine with an API key.
Run VCL VibeBench locally (BYOK)
VCL = Vibe Coder's Life. BYOK = your OpenRouter key in .env. Pick any model IDs from openrouter.ai/models.
1. Clone and install:
git clone https://github.com/kondasviktor/vcl-ai-model-arena.git
cd vcl-ai-model-arena
cp .env.example .env
npm ci
2. Get an OpenRouter API key at openrouter.ai/keys, add credits, then set:
OPENROUTER_API_KEY=sk-or-v1-...
3. Validate model IDs (optional):
MODELS=anthropic/claude-opus-4.8,moonshotai/kimi-k3 npm run validate:models
4. Run Fun and Dev suites:
MODELS=anthropic/claude-opus-4.8,openai/gpt-5.6-sol,moonshotai/kimi-k3 npm run eval:fun
MODELS=anthropic/claude-opus-4.8,openai/gpt-5.6-sol,moonshotai/kimi-k3 npm run eval:dev
5. Refresh the human summary and open the matrix UI:
npm run results:latest
npx promptfoo view
Cheap sanity check (two models, Fun only):
npm run eval:smoke
How you can help
If VCL VibeBench helps you compare models honestly, we would love a ⭐ on GitHub, a fork for your own suite, and a share with someone still trusting benchmark slides. Subscribe to the newsletter for re-runs when the next model drops — you do not need to live in GitHub to stay in the loop.
What is next
VCL VibeBench 1.0 (Fun + Dev) is only the first tier. Two heavier lanes are already on the roadmap — and we want you in the loop before they ship.
Vibe Vision (1.1) — multimodal perception stress tests for motion and static scenes, plus a live /vibe-vision demo on vibecoderslife.com. Goal: see which models actually notice what is on screen when you vibe-build UIs, not only what they claim on text benches.
Vibe Score (1.2) — a near-zero-cost coding subset inspired by Aider-style checks. Goal: a cheap, honest coding signal you can re-run often without burning a week of API credit — useful when a new model drops mid-week.
Later ideas include agent / MCP scenarios and community prompt packs. We are building this in public on GitHub.
Sources: kondasviktor/vcl-ai-model-arena README / methodology; maintainer Fun + Dev run 2026-07-28 via OpenRouter (moonshotai/kimi-k3 and five frontier peers).
Disclosure: OpenRouter is listed on our Tools page. Some catalog links on /tools may be referral or affiliate links when marked there. This post’s clone-and-run commands use standard GitHub and OpenRouter URLs.
Questions? Get in touch — or subscribe for the next AI news post.