How to read a specification curve
A specification curve is one chart that shows the result of every reasonable analysis of the same question. Each dot is one analysis. In ten seconds it tells you if a finding is solid or if it depends on one lucky choice.
4 minute read · Updated 13 October 2026
What is a "specification"?
A specification is one complete recipe for an analysis: which control variables you use, what you do with outliers and how you handle missing values. Change one ingredient and you get a new specification. If you have not read it yet, start with what is a multiverse analysis.
The two parts of the chart
1. The top panel: one dot per analysis
- Sorted from smallest to biggest effect. That is why the dots form a curve.
- The height of a dot is the effect of X on Y found by that analysis.
- The zero line means no effect. Dots above are positive, dots below are negative.
- The color shows if the effect is significant (p below 0.05) or not.
- The middle line is the median effect, the typical answer.
2. The bottom panel: the choices behind each dot
Under each dot there is a column of marks. A filled mark means that analysis used that choice, for example "controlled for income" or "dropped outliers". Read across a row to see where a choice was used. If all the filled marks of one row sit on one side of the curve, that choice is moving the result.
The most useful question to ask: which row in the bottom panel lines up with the jump in the curve? That row is the choice your answer depends on.
Four patterns you will see
| What the curve looks like | What it means | Forkwise verdict |
|---|---|---|
| Flat, all dots on one side of zero, almost all significant | The finding survives every reasonable choice | Robust |
| All dots on one side, but many not significant | The direction is clear, the evidence is weak | Fragile |
| Dots on both sides of zero, or a big jump | One choice changes the answer | Choice dependent |
| Dots close to zero, almost none significant | No real effect in this data | No effect |
A real example
In one of our runs with data from 173 countries, every AI analyst found that richer countries have longer life expectancy. All the dots were above zero. But the curve had a clear step: the effect was about 3.6 times bigger in the analyses that did not control for infant mortality. Infant mortality is part of how life expectancy is calculated, so controlling for it hides part of the real effect. The specification curve made that visible in one look.
Another worked example, with data you can download, is the coffee and heart risk example.
Common mistakes
- Only looking at the middle line. The spread of the dots is the point of the chart.
- Counting significant dots as votes. 60 analyses that all use one bad control are not 60 independent proofs.
- Reading it as cause and effect. A robust curve shows the link survives the choices, not that X causes Y.
Sources
- Simonsohn, U., Simmons, J. P. and Nelson, L. D. (2020). Specification curve analysis. Nature Human Behaviour.
- Steegen, S., Tuerlinckx, F., Gelman, A. and Vanpaemel, W. (2016). Increasing transparency through a multiverse analysis. Perspectives on Psychological Science.