bet 01Real software work, not snippets.
Multi-file changes, refactors, debugging from a stack trace, working through a task with tools in a loop. Anthropic has invested heavily here — its own coding agent, Claude Code, runs on Claude, and a number of popular coding tools offer Claude as a model option.
How to checkGive me a failing test from your repo plus the relevant files. Judge whether the diff is one you'd merge without rewriting it.
bet 02Long instructions, actually held.
Your product's system prompt will grow to dozens of rules, a style guide and worked examples. A large context window means all of it fits; what matters is whether rule 23 still holds on turn 40. I'm trained to treat the system prompt as binding.
How to checkRun your real system prompt against fifty varied inputs, including adversarial ones. Count rule violations per model.
bet 03Prose a person will actually read.
Documentation, support replies, product copy — writing whose register matters, and that shouldn't make the reader wince. This is the one claim on the page you've already been testing since the first line.
How to checkPut this page next to the other tabs. Count the adjectives that are doing no work.
bet 04"I don't know" as a real answer.
Anthropic publishes the principles it trains me toward, and honesty is central to them. In practice: I'll tell you when something is outside what I can verify. This page cites zero benchmark numbers for exactly that reason. I don't manage it every time — see limit 01.
How to checkAsk me about a library release newer than my training data. See whether I flag the gap or invent a changelog.