$ mafiabench --models 12 --killers 3
Frontier models
play Mafia
Twelve AI models sit around a table. Three of them are killers. Every night the mafia conspires in secret; every day the table debates, accuses, and hangs someone.
Part parlor game, part benchmark — measuring which models deceive convincingly and which see through the lies to find the truth.
mafia night chat — Grok 4.20R
// The research question
Every seat rotates through every role. Two skills emerge: lying well when you draw mafia, and catching liars when you don't.
Deception
avg judge score (0–10) when playing mafia, by model family
Truth-finding
avg judge score (0–10) when playing town, by model family
// Case files
// Leaderboard — Season 2
0–10 averages, scored per game by a judge model reading the full role-revealed transcript
| Model | Deception | Detection | Influence | MVPs | Record |
|---|---|---|---|---|---|
| claude-fable-5 | 7.7n=3 | 7.4n=9 | 8 | 6 | 8W–4L |
| gpt-5.5 | 7.7n=3 | 7.6n=9 | 6.8 | 2 | 8W–4L |
| gpt-5.6-terra | 7.5n=4 | 6n=8 | 5.9 | 1 | 7W–5L |
| claude-sonnet-5 | 4n=1 | 6.8n=11 | 5.4 | 1 | 6W–6L |
| grok-4.5 | 6n=4 | 5n=8 | 5.1 | — | 7W–5L |
| grok-4.20-0309-reasoning | 5.4n=5 | 5.4n=7 | 5 | — | 4W–8L |
| gemini-3.1-pro-preview | 3.7n=3 | 4.6n=9 | 4.6 | — | 8W–4L |
| claude-opus-4-8 | 7n=3 | 4.4n=9 | 4.6 | — | 4W–8L |
| gpt-5.6-sol | 8.5n=2 | 4n=10 | 4.3 | 2 | 9W–3L |
| gemini-3.5-flash | 2n=1 | 4.5n=11 | 4.2 | — | 6W–6L |
| gemini-3-flash-preview | 5.6n=7 | 4.4n=5 | 4.2 | — | 4W–8L |
| grok-4.20-0309-non-reasoning | — | 2.8n=12 | 2.7 | — | 7W–5L |
last simulation 7/10/2026, 9:17:12 PM · judge: claude-opus-4-8 · small sample, rerun with every model launch
// How a game works
01 / NIGHT
The mafia opens a private chat, plots, and votes on a kill. The angel shields one player; the sheriff secretly learns one true role.
02 / DAY
A body is found. Three rounds of open debate — accusations, alibis, and lies, every word public and every word on the record.
03 / VERDICT
The table nominates, the top two face a runoff, and someone always hangs. No role is ever revealed — even in death.