← Divination

How scoring works

Divination uses time-weighted proper scoring rules — the Brier score and its continuous cousin, the CRPS — the same family of rules used by forecasting platforms like Metaculus. They have one goal: reward people for being right, early, and honestly calibrated.

The basics

  • Guessing scores around 0 points: the maximally uninformative prediction — 50% on a yes/no, an even split, a spread across the whole plausible range — averages 0, the same as not predicting at all.
  • A perfectly confident, correct prediction earns +100 points.
  • A maximally confident, wrong prediction costs up to −300 points. Overconfidence is punished more than confidence is rewarded — that's what makes honest probabilities your best strategy.

Time matters

Your score isn't taken from your final answer alone — it's averaged over the whole time the question was open. Before your first prediction you sit at the neutral 0, so predicting on day one of a ten-day question puts ten days of your accuracy on the clock, while sniping in the last hour earns almost nothing. You can revise your prediction whenever you like; each revision starts a new segment of the average.

Yes / no questions

You give the probability p that the answer is yes. The instantaneous score is 100 × (1 − 4(p − outcome)²), where the outcome is 1 for yes and 0 for no. Saying 90% and being right earns +96; saying 90% and being wrong costs −224.

Multiple choice

You spread 100% across the options. The score generalises the yes/no rule: your squared error is measured across all options and compared against the uniform "no idea" baseline, scaled so that certainty on the right answer is +100 and the baseline is 0. With two options it behaves exactly like a yes/no question; with more options the worst possible score is milder than −300.

Numeric questions

You predict a distribution, not a single number: your best estimate plus an uncertainty — the ± you'd give even odds of the answer landing within. It's scored with the continuous ranked probability score(CRPS), the Brier score of every implied yes/no question "will the answer be at most x?", added up across all thresholds. The score is 100 × (1 − CRPS / (R/6)) where R is the width of the creator's plausible range— R/6 is what spreading your prediction evenly across the whole range would average, so "I only know the range" sits near 0. A sharp interval that contains the answer scores toward +100; a sharp interval that misses loses big (floored at −300); a wide interval is a safe middle. Because the rule is proper, your expected score is highest when you report your true estimate and your true uncertainty — shading either costs you. Estimating outside the range is allowed.

Date questions

You predict the calendar date something happens, plus an uncertainty in days — even odds it happens within ± that many days of your date. Scoring works exactly like numeric questions with everything measured in days: nailing the date with a tight interval approaches +100, a confident miss bottoms out at −300, and honest uncertainty about the timing is the best strategy. Predicting outside the range is allowed.

Strategy tips

  • Predict early, update often.
  • State your true belief — the scoring rule is designed so shading your probability toward safety always costs you in expectation.
  • On numeric and date questions, your uncertainty is part of the prediction: widen the interval when you genuinely don't know, tighten it when you do.
  • Skipping a question you know nothing about is fine; it scores nothing either way.