Behavioral & Survey Research
Years ago I ran an experiment: people read about crime in a city described either as a beast preying on it or a virus infecting it, then chose how to respond. The wording shifted their policy choices. But "shifted" is a causal claim, and causal claims are hard. This walks through how you actually earn one.
Inspired by Thibodeau & Boroditsky (2011), "Metaphors We Think With," PLoS ONE. Numbers here are illustrative, the method is the point.
Every person has two potential outcomes: what they'd choose under "beast" and what they'd choose under "virus." Causation is the difference between them. The trouble: each person reads only one version, so one of those two is forever unobserved. Reveal the hidden column to see the counterfactual we never get in real life.
Suppose you skipped the experiment and compared people who naturally talk about crime as a beast vs. a virus. Political leaning drives both which metaphor someone favors and how punitive they are, so the difference you measure is part metaphor, part politics. That's confounding. Slide how lopsided the groups are by party.
Assign the frame by a coin flip and the confounder loses its grip: liberals and conservatives end up split roughly evenly across both arms, so the two groups are alike in every way except the metaphor. Now any difference in their choices is caused by the wording. Randomize the sample and watch the party mix balance out, and steady as the sample grows.
With a clean randomized comparison, the estimate is just the difference in the share choosing enforcement: beast minus virus. But an estimate without uncertainty is a guess in a lab coat. The 95% confidence interval shows the range consistent with the data, and it tightens as the sample grows. The red line is "no effect"; the gray marker is the partisan gap, for scale.
What a typical engagement looks like, end to end.