Forecasting · principle
Deterministic Forecasting vs an LLM: Why No Model Touches the Numbers
For forecasts that people act on, the numbers should come from deterministic code, not from a language model. A Monte Carlo simulation with a fixed seed gives the same answer every time, can be back-tested against what actually happened, and shows an honest range instead of a single confident figure. A language model can explain the result; it should not produce it.
01Why not let a model produce the numbers
- Not reproducible. Ask twice, get two answers; an auditor cannot rerun it.
- Not testable the way a model is. You cannot back-test a number that changes with the prompt.
- Confident and wrong. A model will produce a smooth figure even when the data is thin.
02What deterministic forecasting looks like
The system I built for comparing teams and predicting the effect of roster changes works this way. It models each performer separately, then sums to the team. A fixed seed makes every run reproducible, attrition uses smoothing so that zero exits in a small sample is not read as zero risk, and a team budget acts as a shared ceiling.
03Controls that make it trustworthy
| Control | What it does |
|---|---|
| As-of-date back-test | Predicts a past period from data known then, compares with the fact |
| Versioned runs | Each run is stored with a fingerprint of its input data |
| Risk rules | Flags concentration, drawdown and unconfirmed spend |
| Unclosed months flagged | Incomplete data is shown, not silently used |
| Advisory only | The decision stays with a person |
04A worked picture: why the range matters
Suppose a team of ten performers and a plan to add two more. A single-number forecast says 'plus 18 percent' and invites a decision. A simulation replays the next months thousands of times with attrition and uneven returns, and reports a range: a likely band, a pessimistic edge and an optimistic one.
The decision-maker sees how wide the uncertainty is and what drives it. That is the difference between a forecast that supports a decision and one that only decorates it.
05Where a language model still helps
Around the numbers, not inside them: explaining a result in plain language, drafting a summary for a meeting, answering a question about a report. The rule is simple: the model may describe the number; it may not make it.
FAQShort answers
- What is a Monte Carlo simulation in forecasting?
- It runs a model many times with randomised inputs drawn from observed data, and reports the spread of outcomes instead of one number.
- Why use a fixed seed?
- So that the same input always produces the same output, which makes results reproducible and testable.
- Can an LLM be used in forecasting at all?
- Yes, to explain or summarise results, but not to compute them.
- Who builds this?
- I build forecasting systems of this kind for business teams; see the forecasts and dashboards service.
Have a task like this? Write to me.