Does coffee raise heart risk? A hidden variable example
In this dataset, people who drink more coffee have a higher heart risk. But coffee does nothing. The real cause is hiding in another column. This example shows how one analysis choice can create an effect that is not there, and how a multiverse analysis catches it.
4 minute read · Updated 13 October 2026
The data
We made a teaching dataset of 500 adults. Because we made it ourselves, we know the truth: heart risk depends on age, smoking, exercise and BMI, and not on coffee. But smokers in this data also drink more coffee. The correlation between coffee and cigarettes is 0.60.
| Column | Meaning |
|---|---|
| age | Age in years, 30 to 74 |
| coffee_cups_per_day | Cups of coffee per day |
| cigarettes_per_day | Cigarettes per day, 0 for non-smokers |
| exercise_hours_per_week | Hours of exercise per week |
| bmi | Body mass index, 15 rows are empty |
| heart_risk_score | The outcome we want to explain |
Download the data (CSV, 14 KB) and try it yourself.
Five analysts, five answers
Here is what a simple regression says about coffee when you make different choices about control variables. The effect is the change in heart risk score for one more cup per day.
| Controls used | Effect of coffee | p value | Conclusion |
|---|---|---|---|
| None | +1.68 | < 0.001 | Coffee is dangerous |
| Age | +1.75 | < 0.001 | Coffee is dangerous |
| Cigarettes | −0.26 | 0.41 | No effect |
| Age and cigarettes | −0.07 | 0.76 | No effect |
| All four | +0.02 | 0.91 | No effect |
The only choice that matters is whether you control for smoking. Without it, coffee takes the blame for the cigarettes. Scientists call smoking a confounding variable: it affects both coffee drinking and heart risk.
What Forkwise found
We uploaded the file to Forkwise and asked: "Does drinking more coffee increase heart risk?" with 10 AI analysts. Every analyst controlled for cigarettes. They made different choices about outliers and missing BMI values.
Significant analysts: 0 out of 10
Effects found: between −0.04 and +0.05 (per standard deviation of coffee)
p values: between 0.82 and 0.95
Verdict: No effect
The AI analysts did not fall into the trap. But look at the table above: a single analyst who forgot smoking would have reported a strong, highly significant effect. A multiverse analysis shows both paths side by side, so you can see which choice the answer depends on.
What to learn from this
- A small p value is not proof. The first two rows have p below 0.001 and are still wrong.
- Ask what else could explain the link. Is there a column that affects both X and Y?
- Look at many analyses, not one. If the result flips when you add one control, report that.
- A hidden variable outside your file can still fool every analysis. If smoking was not in the data, all analysts would agree on the wrong answer.
Try a second example
Download the study hours data (CSV, 10 KB): 400 students with weekly study hours, sleep, attendance and exam scores. Here more study really does raise scores, so the verdict should be robust. Three students wrote impossible study hours on purpose. Watch how the effect size changes when an analyst removes these outliers.