
MALI Platform
AI Stock Research
Brier Score Calculator
Paste the probabilities you gave and what happened. You get the same measures MALI's public track record uses: the Brier score, skill against always saying 50%, how often your calls leaned the right way, and whether your 70% calls happened about 70% of the time.
Probabilities can be 0.7, 70 or 70%; outcomes 1/0, yes/no or true/false. The lines above are an example.
Forecasts scored10
Brier score (0 is perfect, 0.25 is always saying 50%)0.168
Skill against always saying 50%+32.9%
Calls that leaned the right way70.0% (7/10)
95% interval for that rate39.7% – 89.2%
Calibration: when you said X%, how often did it happen?
| Band | n | Said | Happened | 95% interval |
|---|---|---|---|---|
| 20–30% | 1 | 20% | 0% | 0%–79% |
| 30–40% | 1 | 30% | 0% | 0%–79% |
| 40–50% | 1 | 40% | 100% | 21%–100% |
| 50–60% | 1 | 55% | 100% | 21%–100% |
| 60–70% | 2 | 63% | 0% | 0%–66% |
| 70–80% | 2 | 73% | 100% | 34%–100% |
| 80–90% | 1 | 80% | 100% | 21%–100% |
| 90–100% | 1 | 90% | 100% | 21%–100% |
With fewer than 30 forecasts the intervals are wide; treat the scores as a first look, not a verdict.
How it works
- 1The Brier score is the average of (probability − outcome)², with the outcome 1 if it happened and 0 if it didn't. Zero is perfect; always saying 50% scores 0.25.
- 2Skill compares your score with that 0.25: above zero you did better than a coin flip, below zero worse.
- 3The hit rate counts a forecast above 50% as a call that it would happen, and one below 50% as a call that it wouldn't, with a Wilson 95% interval. Calibration groups forecasts into bands and compares what you said with how often it happened.
Questions
- How many forecasts do I need?
- As many as you can. Under about 30 the intervals are wide and one surprise moves the score a lot; the calculator says so.
- Is a high hit rate enough?
- No. A forecaster who says 99% and is right 80% of the time has a good hit rate and poor calibration. The Brier score and the calibration table catch that.
- How does MALI use this?
- Its public track record scores every daily call this way, against SPY, and draws no conclusion below 30 scored calls.