50games
9,484decisions put to a model
23,911moves played
14.23Mtokens
$5.54logged cost
3.3hwall clock
standings
| model | wins | rate | interval (adj. 4 models) | median | p90 | slowest | failed | rate | out/call | cost |
|---|
| deepseek-v4-flash | 19/50 | 38% | 23.1–55.6% | 49.1s | 174.4s | 1866s | 3,635 | 60% | 1899 | $1.27 |
| gpt-5.6-luna | 19/50 | 38% | 23.1–55.6% | 5.4s | 8.6s | 31s | 2 | 0% | 302 | $1.23 |
| gemini-3.7-flash | 7/50 | 14% | 5.8–30.2% | 2.9s | 4.2s | 26s | 3 | 0% | 186 | $1.43 |
| hy3 | 5/50 | 10% | 3.5–25.4% | 13.9s | 41.8s | 137s | 591 | 21% | 1107 | $1.62 |
Chance is 25% with 4 seats. Intervals are Bonferroni-adjusted — testing every model against chance is several tests, and an unadjusted 95% interval crowns a winner about one run in four by luck alone. Timings here are whole-decision wall clock including retries and backoff, not model latency — these logs predate the split, so a model that retried often reads far slower than it is.
per model
median seconds per decision, retries included
deepseek-v4-flash49.1s
hy313.9s
gpt-5.6-luna5.4s
gemini-3.7-flash2.9s
failed attempts
deepseek-v4-flash60%
hy321%
gemini-3.7-flash0%
gpt-5.6-luna0%
output tokens per decision
deepseek-v4-flash1899
hy31107
gpt-5.6-luna302
gemini-3.7-flash186
cost
hy3$1.62
gemini-3.7-flash$1.43
deepseek-v4-flash$1.27
gpt-5.6-luna$1.23
failures
4,231 failed attempts against 9,484 successful decisions. Every one was retried until the model answered — no move was ever substituted.
| failure | count | share |
|---|
| timed out on our deadline | 4,188 | 99% |
| unparseable reply | 41 | 1% |
| connect error | 1 | 0% |
| remoteprotocol error | 1 | 0% |
retry depth
deepest chain: 17 attempts
| attempt 1 | 2,298 |
| attempt 2 | 1,144 |
| attempt 3 | 490 |
| attempt 4 | 201 |
| attempt 5 | 56 |
| attempt 6 | 16 |
| attempt 7 | 9 |
| attempt 8 | 7 |
seat balance
Turn order is worth real points, so seats rotate per match. These should be even.
| model | seat 1 | seat 2 | seat 3 | seat 4 |
|---|
| deepseek-v4-flash | 13 | 12 | 12 | 13 |
| gemini-3.7-flash | 12 | 13 | 13 | 12 |
| gpt-5.6-luna | 12 | 12 | 13 | 13 |
| hy3 | 13 | 13 | 12 | 12 |
game length
median 148 turns, range 58–355; median 450 moves. Only 40% of moves went to a model — forced moves with a single legal action skip the API.
every record
Each game replays move by move from its own log. Raw files are served as text.