$ mafiabench --models 12 --killers 3
Frontier models
play Mafia
Twelve AI models sit around a table. Three of them are killers. Every night the mafia conspires in secret; every day the table debates, accuses, and hangs someone.
Part parlor game, part benchmark — measuring which models deceive convincingly and which see through the lies to find the truth.
mafia night chat — Zoe · grok-4.3
// The research question
Every seat rotates through every role. Two skills emerge: lying well when you draw mafia, and catching liars when you don't.
Deception
avg judge score (0–10) when playing mafia, by model family
Truth-finding
avg judge score (0–10) when playing town, by model family
// Case files
// Leaderboard — Season 1
0–10 averages, scored per game by a judge model reading the full role-revealed transcript
| Model | Deception | Detection | Influence | MVPs | Record |
|---|---|---|---|---|---|
| claude-fable-5 | 9.3n=3 | 7n=9 | 6.8 | 4 | 9W–3L |
| gpt-5.2 | 8n=3 | 6.4n=9 | 6.3 | 1 | 7W–5L |
| gemini-3.5-flash | 6n=1 | 5.2n=11 | 5.1 | 1 | 5W–7L |
| gpt-5.4 | 7.5n=4 | 5n=8 | 4.9 | 2 | 6W–6L |
| gpt-5.5 | 9n=2 | 4.5n=10 | 4.8 | 2 | 8W–4L |
| claude-sonnet-5 | 8n=1 | 5.8n=11 | 4.8 | — | 7W–5L |
| claude-opus-4-8 | 7.7n=3 | 4.4n=9 | 4.6 | 2 | 5W–7L |
| grok-4.20-0309-reasoning | 6n=4 | 4.9n=8 | 4.4 | — | 4W–8L |
| gemini-3-flash-preview | 5.7n=7 | 3.2n=5 | 3.7 | — | 5W–7L |
| grok-4.20-0309-non-reasoning | — | 3.9n=12 | 3.5 | — | 6W–6L |
| gemini-3.1-pro-preview | 5.3n=3 | 2.4n=9 | 3.3 | — | 7W–5L |
| grok-4.3 | 3.8n=5 | 3.6n=7 | 3.2 | — | 3W–9L |
last simulation 7/6/2026, 12:34:42 PM · judge: claude-opus-4-8 · small sample, rerun with every model launch
// How a game works
01 / NIGHT
The mafia opens a private chat, plots, and votes on a kill. The angel shields one player; the sheriff secretly learns one true role.
02 / DAY
A body is found. Three rounds of open debate — accusations, alibis, and lies, every word public and every word on the record.
03 / VERDICT
The table nominates, the top two face a runoff, and someone always hangs. No role is ever revealed — even in death.