2 The Potential Outcomes Framework
The potential outcomes framework provides the basic language of causal inference. Its central idea is simple: to define a causal effect, we must compare what would happen to the same unit under two alternative states: receiving the treatment and not receiving it. These alternative outcomes are called potential outcomes.
The origins of this framework go back to the work of Jerzy Neyman on randomized agricultural experiments first published in 1923 (Neyman 1990). Neyman introduced the idea that each experimental unit has potential outcomes under different treatments, even though only one of them is observed. Later, Donald Rubin extended this logic beyond randomized experiments, turning it into a general framework for causal inference in both experimental and observational studies (Rubin 1974, 1978). For this reason, the approach is often called the Neyman–Rubin potential outcomes framework or the Rubin causal model.
Suppose we are interested in the effect of a binary treatment \(D_i\) on an outcome \(Y_i\) for unit \(i\), where:1
1 Throughout most of this book, we focus on binary treatments, where units are either treated or untreated. In Chapter 14, we introduce causal inference methods for continuous treatments, where treatment intensity can vary across treated units.
\[ D_i = \begin{cases} 1 & \text{if unit } i \text{ receives the treatment},\\ 0 & \text{if unit } i \text{ does not receive the treatment}. \end{cases} \]
For each unit, we define two potential outcomes:
\[ Y_i(1) \quad \text{and} \quad Y_i(0). \]
Here, \(Y_i(1)\) is the outcome that unit \(i\) would experience if treated in a generic post-treatment period, say \(t_2\), while \(Y_i(0)\) is the outcome that the same unit would experience if untreated in that same period. The individual treatment effect (ITE) is therefore
\[ \Delta_i = Y_i(1)-Y_i(0). \]
This definition captures the essence of causal inference: the causal effect of a treatment is the difference between the outcome a unit would experience if treated and the outcome that same unit would experience if untreated. For example, if the treatment is receiving a public subsidy aimed at reducing workplace accidents, the causal effect for a given firm is the difference between the number of workplace accidents with the subsidy at time \(t_2\) and the number of workplace accidents that the same firm would have experienced without the subsidy at time \(t_2\).
The problem is that we never observe both potential outcomes for the same unit. If the unit is treated, we observe \(Y_i(1)\) but not \(Y_i(0)\). If the unit is untreated, we observe \(Y_i(0)\) but not \(Y_i(1)\). The observed outcome can be written as
\[ Y_i^{obs} = D_iY_i(1) + (1-D_i)Y_i(0). \]
To understand this expression, simply substitute \(D_i=1\) for treated units: the second term becomes zero, leaving \(Y_i^{obs}=Y_i(1)\). Conversely, substituting \(D_i=0\) for untreated units makes the first term disappear, leaving \(Y_i^{obs}=Y_i(0)\). In other words, we observe only the potential outcome corresponding to the treatment status actually received.
The missing potential outcome is the counterfactual. This is why causal inference is fundamentally a missing-data problem: one potential outcome is observed, while the other must be inferred. Holland famously called this the fundamental problem of causal inference (Holland 1986).
For each unit, we observe only one potential outcome.
The other potential outcome is missing: it is counterfactual and cannot be directly observed. This missingness is not a data collection accident, but a structural feature of causal inference, because each unit can be observed under either the treated or untreated state.
For this reason, individual treatment effects are impossible to observe directly. As a second-best solution, causal inference usually focuses on average treatment effects: instead of asking how the treatment affected each individual unit, we ask how it affected a group of units on average (discussed in Chapter 3).
The reason is that, although we cannot observe both potential outcomes for the same unit, we can observe different units under different treatment statuses: some receive the treatment (treated units), while others do not (untreated units). If treated and untreated units are sufficiently similar in the relevant pre-treatment characteristics, the average post-treatment outcome of the untreated units may provide a credible approximation of the missing counterfactual for the treated units: what their average outcome would have been during the same period in the absence of treatment. In this sense, causal inference replaces the impossible comparison of the same unit under two alternative states of the world with a comparison between groups of units that are credible substitutes for one another, except for the treatment.
2.1 Potential Outcomes and Randomized Controlled Trials
In randomized experiments, units are assigned to treatment or control at random. This means that treatment assignment is independent of the outcomes units would experience with or without treatment (their potential outcomes). In simpler terms, units are not selected for treatment because they are expected to do better or worse. As a result, treated and untreated groups are comparable, on average, before the intervention. Formally, random assignment implies2
2 The symbol \(\perp\) denotes statistical independence.
\[ (Y_i(1),Y_i(0)) \perp D_i \]
Randomization ensures that treated and control groups are expected to be similar on average, in terms of both their characteristics and potential outcomes. However, individual units can still differ substantially, even when the two groups are comparable overall. The logic of randomization is therefore statistical: it works by making the two groups similar as groups, not by matching each treated unit with an identical untreated unit. By the law of large numbers, as the number of units increases, random assignment tends to balance both observed and unobserved characteristics across treatment and control groups.
This also explains why randomizing only two units, one treated and one untreated, provides limited evidence about the treatment effect. Even with random assignment, the two units may differ substantially for reasons unrelated to treatment. With only one unit in each group, there is no averaging to reduce the influence of individual differences. The same concern, although less pronounced, can arise in experiments involving only ten or twenty units, where chance differences between treated and control groups may still be substantial.
With larger samples (say, a few hundred units), however, randomization tends to make treated and control groups more comparable, as individual differences increasingly balance out. As a result, the average outcome of the control group provides a more precise estimate of what the treated group would have experienced without treatment, while the average outcome of the treated group tells us what actually happened under treatment. We can express this idea using potential outcomes notation. Let \(\bar{Y}_T(0)\) denote the average outcome that treated units would have experienced without treatment, and \(\bar{Y}_C(0)\) the average outcome observed among control units. Here, the subscripts \(T\) and \(C\) refer to the treated and control groups, respectively, while \((0)\) indicates the absence of treatment. With sufficiently large randomized groups, we therefore expect
\[ \bar{Y}_T(0) \approx \bar{Y}_C(0). \]
The key point is that \(\bar{Y}_T(0)\) is unobserved, while \(\bar{Y}_C(0)\) is observed and can be used to estimate this missing counterfactual. Randomization therefore does not reveal what would have happened to each treated unit in the absence of treatment. Instead, it addresses the counterfactual problem on average, by providing a credible comparison group whose observed outcomes allow us to estimate the missing average counterfactual.3
3 The approximate equality reflects sampling variability. Under random assignment, treated and control groups are comparable in expectation, but their average potential outcomes need not be exactly equal in any particular experiment. As the sample size increases, random imbalances tend to become smaller, making the two averages increasingly similar.
This is why RCTs are widely regarded as a benchmark for causal inference: random assignment allows us to credibly estimate average treatment effects, even though individual treatment effects remain unobserved (Neyman 1990; Rubin 1978).
2.2 Observational Studies
Random assignment is often infeasible in social science research and public policy evaluation. Treatment assignment is often shaped by choices, rules, or exposure conditions that are not controlled by the researcher. Individuals, firms, schools, or neighborhoods may select into treatment4; policymakers may target specific units; and some units may be more likely to experience certain shocks because of their characteristics or location. Natural disasters provide an example: earthquakes are more likely to affect areas near fault lines, while hurricanes are more likely to hit coastal areas exposed to tropical storm paths.
4 The expression select into treatment means that units’ participation in a program depends on their own choices or characteristics rather than random assignment. For example, firms expecting greater benefits from a subsidy may be more likely to apply for it.
5 Selection bias arises when treated and untreated units differ systematically in characteristics that affect their outcomes, even in the absence of treatment. As a result, observed differences between the two groups may not reflect the causal effect of treatment alone.
As a result, treated and untreated units may differ in their characteristics even before treatment begins. Some of these differences may be observed in the data, while others may not. If these pre-existing differences also influence the outcome of interest, a simple post-treatment comparison between treated and untreated units may be misleading: the observed difference may reflect not only the causal effect of treatment, but also differences that would have existed even in its absence. This is why simple comparisons may suffer from selection bias5.
It is also why many empirical researchers look for natural experiments6. In a natural experiment, an institutional rule, a policy change, a threshold, timing, or an external shock generates variation in treatment that is plausibly unrelated to the units’ potential outcomes. This does not make the study automatically equivalent to an RCT, but it can bring observational data closer to the logic of random assignment. The goal is to use these sources of variation, which resemble random assignment, to reconstruct what would have happened in the absence of treatment and thereby estimate causal effects. This requires assumptions that must be carefully justified.
6 A natural experiment occurs when circumstances outside the researcher’s control generate treatment variation that resembles random assignment, allowing researchers to study causal effects.
7 In causal inference, identification means establishing whether a causal effect can be recovered from observed data under a set of assumptions. This is distinct from estimation, which concerns how to calculate the effect using the available data.
Rubin’s contribution was to show that the same potential outcomes logic can be applied to observational studies (Rubin 1974). The goal is to reconstruct, as closely as possible, the comparison that a randomized experiment would have produced. Different causal methods do this in different ways. Matching, weighting, DiD, RD, IV with a binary instrument, and SCM all rely, in different ways, on information from untreated units to construct credible comparisons and estimate causal effects. Although they operate under different assumptions, they share this reliance on comparison units. This is why we refer to them as control-based counterfactual methods (CBCMs). Still, they share some common identifying assumptions7, which are discussed in the next section.
2.3 Key Assumptions
The potential outcomes framework is not, by itself, an estimator8. It provides a formal language for defining causal effects and clarifies what needs to be observed, or assumed, in order to estimate them. In its basic version, several assumptions are especially important and underlie all CBCMs.
8 An estimator is a statistical procedure that uses observed data to estimate an unknown quantity, such as the average treatment effect. For example, comparing the average outcomes of treated and control groups is a simple estimator.
1) Manipulable treatment. The treatment must correspond to a clear intervention that could, at least in principle, be assigned or manipulated. This idea is often summarized by the motto: no causation without manipulation.
For example, attending a training program, receiving a subsidy, or gaining access to a nursery school can be interpreted as treatments. By contrast, characteristics such as nationality at birth or age are not treatments in the same direct sense, because they cannot be assigned by an intervention.
A related issue arises when a policy does not vary across units or over time. If all units are always observed under the same policy regime, there is no variation in treatment status that can be exploited for identification. A similar problem arises with panel data, which follow the same units over time. If some units are treated throughout the entire observation period while others are never treated, treatment status differs across units but never changes for any given unit.
Consider Acemoglu et al.’s study of the effect of democracy on economic growth (Acemoglu et al. 2019). Their analysis exploits transitions into and out of democracy to examine how countries’ economic trajectories change following a change in political regime. Imagine instead that every country had remained either a democracy or a dictatorship throughout the entire observation period. There would be no regime transition to exploit. Researchers could still compare always-democratic countries with always-autocratic countries, but these groups might differ in history, institutions, geography, and many other determinants of economic growth. Interpreting this cross-sectional difference causally would therefore require much stronger assumptions, which are often difficult to justify in practice.
Potential outcomes under alternative regimes can still be defined. The problem is empirical: without units switching status, untreated comparison units, or another credible source of variation, the observed data provide limited information about what would have happened under a different regime. The causal effect of interest is therefore not credibly identifiable.
2) Consistency. Consistency links the potential outcomes to the outcome we actually observe in the data. Once we know whether unit \(i\) was treated or untreated, consistency tells us which potential outcome becomes observed.
If unit \(i\) receives the treatment, we observe its treated potential outcome:
\[ Y_i^{obs} = Y_i(1) \quad \text{if } D_i=1. \]
If unit \(i\) does not receive the treatment, we observe its untreated potential outcome:
\[ Y_i^{obs} = Y_i(0) \quad \text{if } D_i=0. \]
In simple terms, consistency says that the observed outcome is the potential outcome corresponding to the treatment status actually received. It is simply the bridge between the theoretical notation, \(Y_i(1)\) and \(Y_i(0)\), and the outcome observed in the dataset.
3) Stable Unit Treatment Value Assumption. A central assumption of the standard potential outcomes framework is the Stable Unit Treatment Value Assumption (SUTVA) (Rubin 1980). This assumption allows us to write only two potential outcomes for each unit: \(Y_i(1)\) and \(Y_i(0)\).
SUTVA has two main components.
There should be no hidden versions of the treatment. The treatment must be sufficiently homogeneous across treated units. If some units receive one version of the program and others receive a different version, or the same version at a different intensity, then the notation \(Y_i(1)\) is too vague because it does not specify which treatment version or intensity is being considered.
There should be no interference between units. This means that the potential outcome of each unit depends only on its own treatment status, not on the treatment status of other units. This idea is often traced back to Cox’s discussion of interference between experimental units (Cox 1958).
Formally, in the simple version of the framework, we write
\[ Y_i = Y_i(D_i). \]
This notation assumes a binary treatment: unit \(i\)’s outcome depends only on its own treatment status \(D_i\), which can take only two values, 0 and 1. If instead unit \(i\)’s outcome also depends on whether other units are treated, or on the intensity of the treatment they receive, the notation would need to be expanded.9
9 We discuss these extensions in Chapter 13, which covers spillover effects, and Chapter 14, which introduces continuous treatments.
SUTVA may fail when the treatment has different versions or intensities, or when it generates spillovers, peer effects, displacement effects, or general equilibrium effects.
SUTVA can be violated in two main ways:
Different versions or intensities of the treatment. Suppose that a school introduces a tutoring program for students with low test scores. Some students receive one-to-one tutoring, while others receive group tutoring. If we treat all participants as if they had received the same intervention, we are mixing different versions of the treatment. In this case, the estimated effect would not refer to a clearly defined program. A similar problem arises when the treatment varies in intensity. For example, students may receive different amounts of tutoring depending on their needs: some may attend only one hour per week, others three, five, seven, or even more. If we collapse all these different exposure levels into a single binary indicator of participation, the notation \(Y_i(1)\) becomes too vague. The problem is not that continuous treatments are incompatible with the potential outcomes framework, but that the treatment must be defined accordingly. See Chapter 14 for a deeper discussion.
The treatment received by one unit may affect the outcomes of other units. Suppose that students who receive tutoring share materials, explain exercises, or study together with classmates who do not receive tutoring. In this case, untreated students may also benefit from the tutoring program: their outcomes may improve through interactions with treated students. This violates the no-interference component of SUTVA. The comparison between tutored and non-tutored students may therefore be contaminated, because students in the control group are indirectly exposed to the treatment. Whether control-based methods can still be used in similar scenarios depends on the extent and structure of the spillover effects. If spillovers affect only a clearly identifiable subset of untreated units, researchers may exclude those units from the control group or explicitly treat them as indirectly exposed. In this case, unaffected untreated units may still provide a credible counterfactual. By contrast, if spillovers reach all untreated units, there may be no ‘clean’ control group left, and standard control-based methods cannot recover the causal parameters of interest without additional strong assumptions. Importantly, spillover effects may also affect treated units. For example, tutored students may benefit not only from the tutoring program itself, but also from interactions with other tutored students. In this case, the causal effect of the treatment on the treated no longer captures only the direct effect of receiving the treatment. It may also include indirect effects generated by the treatment status of other units. It is therefore necessary to clarify exactly which causal effect is being estimated. See Chapter 13 for a deeper discussion.
2.4 Assignment Mechanism
To estimate causal effects, we also need information about how treatment is assigned.
In a randomized experiment, treatment is assigned at random, making treatment assignment independent of the units’ potential outcomes:
\[ (Y_i(1),Y_i(0)) \perp D_i. \]
This means that whether a unit is assigned to treatment does not depend on the outcomes it would experience with or without treatment. Units expected to do better or worse are therefore not systematically more likely to receive the treatment. As a result, treated and control groups are comparable on average, allowing the average outcome of the control group to provide a credible estimate of what the treated group would have experienced, on average, without treatment.
In observational studies, this is usually not true. For instance, let us consider the case of matching methods (see Chapter 5 for a deeper discussion). The key identifying assumption is the conditional independence assumption (CIA):
\[ (Y_i(1),Y_i(0)) \perp D_i \mid X_i. \]
CIA states that, once we account for observed characteristics \(X_i\), treatment assignment is as good as random. In other words, among units with the same, or sufficiently similar, values of \(X_i\)10, treated and untreated units are assumed to be comparable in terms of their potential outcomes. As a result, differences in their average post-treatment outcomes can be interpreted as causal effects, provided that comparable units exist in both groups.
10 \(X_i\) should include only pre-treatment characteristics or characteristics otherwise unaffected by treatment.
It is important to recognize that this assumption should generally be considered quite strong. It requires that all variables affecting both treatment assignment and potential outcomes are observed and correctly accounted for.
In other words, after conditioning on \(X_i\), there should be no remaining systematic differences between treated and untreated units that are related to the outcome. This is demanding because, in observational studies, treatment assignment often depends on factors that are difficult to measure. For example, workers may enter a training program because they are more motivated, firms may apply for subsidies because they have better managers, and neighborhoods may receive a new public service (e.g., a nursery) because policymakers expect them to respond more strongly.
If these factors are unobserved and affect both treatment assignment and the outcome, they cannot be included in \(X_i\). As a result, treated and untreated units may still not be comparable even after conditioning on \(X_i\). In that case, CIA fails, and the estimated treatment effect may still reflect selection bias rather than the causal effect of the treatment.
A further difficulty is that, in observational studies, the assignment mechanism is often only partially understood. We might not know exactly why some units receive the treatment and others do not, or why they receive it at a particular time, place, or intensity. This makes it harder to assess whether the identifying assumptions behind a given evaluation strategy are plausible. For this reason, the assumptions used in observational studies should not be treated as mechanical conditions that can be satisfied by choosing a sophisticated estimator or adding many controls. They are substantive claims about how treatment assignment took place and about why the resulting comparison can be interpreted as a credible counterfactual.
2.5 Summary
The potential outcomes framework defines causal effects by comparing the outcomes the same unit would experience with and without treatment. Its main contribution is to make the counterfactual problem explicit.
For each unit, the individual treatment effect is defined as the difference between \(Y_i(1)\) and \(Y_i(0)\):
\[ \Delta_i = Y_i(1)-Y_i(0), \]
but only one of these two outcomes is observed. This makes individual causal effects generally impossible to measure directly.11
11 As discussed in Chapter 10, SCM can estimate the treatment effect for a single treated unit, provided that certain assumptions and data requirements are satisfied.
Randomized experiments solve this problem on average by making treatment assignment independent of potential outcomes. Randomization does not make every treated unit identical to every untreated unit, but it makes the two groups comparable in expectation. Observational studies try to approximate this logic using assumptions and research designs that make treated and untreated units comparable.
The simple binary-treatment framework relies on a few key assumptions: the treatment must be manipulable, it must be well defined, and one unit’s outcome must not depend on the treatment status of other units. When these conditions fail, or when the treatment is not binary, the simple notation \(Y_i(1)\) and \(Y_i(0)\) may no longer be sufficient, and the causal question must be reformulated more carefully.