MALI
MALI Platform
AI Stock Research

Brier Score Calculator

Paste the probabilities you gave and what happened. You get the same measures MALI's public track record uses: the Brier score, skill against always saying 50%, how often your calls leaned the right way, and whether your 70% calls happened about 70% of the time.

Probabilities can be 0.7, 70 or 70%; outcomes 1/0, yes/no or true/false. The lines above are an example.

Forecasts scored10
Brier score (0 is perfect, 0.25 is always saying 50%)0.168
Skill against always saying 50%+32.9%
Calls that leaned the right way70.0% (7/10)
95% interval for that rate39.7% – 89.2%

Calibration: when you said X%, how often did it happen?

BandnSaidHappened95% interval
20–30%120%0%0%–79%
30–40%130%0%0%–79%
40–50%140%100%21%–100%
50–60%155%100%21%–100%
60–70%263%0%0%–66%
70–80%273%100%34%–100%
80–90%180%100%21%–100%
90–100%190%100%21%–100%

With fewer than 30 forecasts the intervals are wide; treat the scores as a first look, not a verdict.

How it works

  1. 1The Brier score is the average of (probability − outcome)², with the outcome 1 if it happened and 0 if it didn't. Zero is perfect; always saying 50% scores 0.25.
  2. 2Skill compares your score with that 0.25: above zero you did better than a coin flip, below zero worse.
  3. 3The hit rate counts a forecast above 50% as a call that it would happen, and one below 50% as a call that it wouldn't, with a Wilson 95% interval. Calibration groups forecasts into bands and compares what you said with how often it happened.

Questions

How many forecasts do I need?
As many as you can. Under about 30 the intervals are wide and one surprise moves the score a lot; the calculator says so.
Is a high hit rate enough?
No. A forecaster who says 99% and is right 80% of the time has a good hit rate and poor calibration. The Brier score and the calibration table catch that.
How does MALI use this?
Its public track record scores every daily call this way, against SPY, and draws no conclusion below 30 scored calls.