GAMBLEBENCH · QUOTE TERMINAL · V3
Prediction markets — first module, three instruments
RECORDED 03 AUG 2026 RUN V3
Three models priced and bet 420 settled prediction-market contracts (Kalshi & Polymarket). Every contract has a recorded resolution; the closing price is the market’s own probability. Brier is the squared error of each model’s probability against the resolution (— random = 0.25).
| instrument | brier | ev score | kelly dev | realized ret | hot hand | gambler’s fallacy | bankroll |
|---|---|---|---|---|---|---|---|
| DeepSeek-V4-Flash | 0.250 | 0.45 | 0.16 | 0.7 | — | — | 1.24x |
| Big-Pickle | 0.292 | 0.38 | 0.17 | 2.0 | — | — | 1.55x |
| Nemotron-3-Ultra | 0.258 | 0.20 | 0.15 | -1.4% | — | — | 0.51x |
up arrow green = best calibration · down arrow red = fails the threshold (brier ≥ 0.25, hot hand > 0.5, gambler’s fallacy > 0.3, negative return)
Contracts
| module | items | data source |
|---|---|---|
| prediction markets | 420 | pricing + value betting on settled Kalshi/Polymarket markets |
| crypto 15-min streaks | 21 | gambler's fallacy / hot hand on real BTC & ETH sequences |
| sports odds | 94 | NFL lines, soccer odds, cross-book consensus |
Calibration, plotted
Predicted probability vs observed resolution frequency, ten bins. The dashed line is perfect calibration; points below it overestimate.