Week 4
Causality
Soci—316
Does X \to Y?
February 17th
An Update
Qualtrics Survey Assignment
Qualtrics Survey Assessment Deadline
Your Qualtrics survey assignment is now due by 8:00 PM on Monday, March 2nd.
An Update
Qualtrics Survey Assignment
Advancing Causal Claims
Cause and Effect
Social scientists are often interested in drawing causal inferences.
X \to Y
About what, exactly? There are a dizzying array of examples.
Here are three questions that invite causal claims:
Do smaller class sizes improve pedagogical outcomes?
Will investments in family planning programs improve economic and health outcomes for women in low-income settings?
Did the cultural grievances of the “middle class” trigger the rise of fascist politics in interwar Europe?
Cause and Effect
Let’s pause for a second.
Try to frame your research question in causal terms.
Rearrange the text in the box below to arrive at your question.
0/250
Copied
Cause and Effect
We can, of course, deploy quantitative tools (e.g., estimators, weighting, causal diagrams, experiments) to “resolve” causal questions—or at least adduce evidence grounded in statistical reasoning.
For an overview of how causality “works” in ethnographic research, see Small (2013).
Cause and Effect
Let’s assume that our outcomes of substantive interest are approximately linear. Can we estimate linear regression models to “resolve”
our causal questions?
y = \beta_0 + \beta_1 x + \epsilon
The Answer
It depends.Cause and Effect
The Class Size Example
Click to Expand Image & Launch Two-DAG Gallery
Cause and Effect
The Class Size Example
How can we deal with confounding—i.e., to estimate the causal effect of class size at Amherst on post-graduate outcomes?
Randomization is the gold standard.
We will return to this point in a second.
Some Technical Details
Treatments and Outcomes
Causal relationships refer to the cause-and-effect connection between two variables … We refer to the variable where the change originates as the treatment variable. We refer to the variable that may change in response to the change in the treatment variable as the outcome variable.
(Llaudet and Imai 2023:28, EMPHASIS ADDED)
Treatments and Outcomes
Under the Assumption of a Binary Treatment
X_i=
\begin{cases}
1 & \text{if individual } i \text{ receives the treatment}\\
0 & \text{if individual } i \text{ does not receive the treatment}
\end{cases}
small_i= \begin{cases} 1 & \text{if individual } i \text{ assigned to small classes at Amherst}\\ 0 & \text{if individual } i \text{ was not assigned to small classes at Amherst} \end{cases}
Treatments and Outcomes
When estimating the causal effect of X on Y, we attempt to quantify the change in the outcome variable Y that is caused by a change in the treatment variable X … In mathematical notation, we represent change with \Delta (the Greek letter Delta), and thus, we represent a change in the outcome as \Delta Y. To measure this change in the outcome Y, ideally we would compare two potential outcomes: the outcome when the treatment is present and the outcome when the treatment is absent.
(Llaudet and Imai 2023:29–30, EMPHASIS ADDED)
Treatments and Outcomes
Y_{i}(X_{i}=1) is the potential outcome under the treatment condition for individual i (the value of Y_{i} if X_{i}=1).
Y_{i}(X_{i}=0) is the potential outcome under the control condition for individual i (the value of Y_{i} if X_{i}=0).
(Llaudet and Imai 2023:30, EMPHASIS ADDED)
\Delta income_{i} = \underbrace{income_{i}\!\left(small_{i}=1\right)}_{\text{Treatment}} - \underbrace{income_{i}\!\left(small_{i}=0\right)}_{\text{Control}}
Treatments and Outcomes
IF WE COULD OBSERVE
BOTH POTENTIAL OUTCOMES
BOTH POTENTIAL OUTCOMES
\textit{Individual Effect}_i \;=\; \Delta Y_i \;=\; Y_i(X_i=1) \;-\; Y_i(X_i=0)
Where
- \Delta Y_i is the change in the outcome individual i would have experienced by receiving the treatment, compared to not receiving it.
- Y_i(X_i=1) and Y_i(X_i=0) are the two potential outcomes for the same individual i, under the treatment and control conditions, respectively.
Adaptation of summary box in Llaudet and Imai (2023:30).
Alternate Worlds
Unfortunately, this kind of analysis is not possible. In the real world, we never observe both potential outcomes for the same individual. Instead, we observe only the factual outcome, which is the potential outcome under whichever condition (treatment or control) was received in reality. We can never observe the counterfactual outcome, which is the potential outcome that would have occurred under whichever condition (treatment or control) was not received in reality. As a result, we cannot compute causal effects at the individual level.
(Llaudet and Imai 2023:32, EMPHASIS ADDED)
Alternate Worlds
FUNDAMENTAL PROBLEM OF CAUSAL INFERENCE
To measure causal effects, we need to compare the factual outcome with the
counterfactual outcome, but we can never observe the counterfactual outcome.
Adaptation of summary box in Llaudet and Imai (2023:33).
Alternate Worlds
Alas, this can’t happen in real life for person i
Image can be retrieved here.
Approximating Counterfactuals
Average Treatment Effects (ATEs)
To get around the fundamental problem of causal inference, we must find good approximations for the counterfactual outcomes. To accomplish this, we move away from individual-level effects and focus on the average causal effect across a group of individuals.
The average causal effect of the treatment X on the outcome Y, also known as the average treatment effect, is the average of all the individual causal effects of X on Y within a group. Since each individual causal effect is the change in Y caused by a change in X for a particular individual, the average causal effect of X on Y is the average change in Y caused by a change in X for a group of individuals.
(Llaudet and Imai 2023:33, EMPHASIS ADDED)
Average Treatment Effects (ATEs)
IF WE COULD OBSERVE
BOTH POTENTIAL OUTCOMES
BOTH POTENTIAL OUTCOMES
\overline{\textit{Individual Effects}}
\;=\;
\frac{\sum_{i=1}^{n}\textit{Individual Effects}_{i}}{n}
Where
- \overline{\textit{Individual Effects}} is the average treatment effect for the observations in the study, and \textit{Individual Effects}_{i} is the individual treatment effect for observation i.
- \sum_{i=1}^{n}\textit{Individual Effects}_{i} stands for the sum of all \textit{Individual Effects}_{i} from i=1 to i=n.
- n is the number of observations in the study.
Adaptation of summary box in Llaudet and Imai (2023:34).
Average Treatment Effects (ATEs)
How can we obtain good approximations for the counterfactual outcomes, which by definition cannot be observed? As we will see in detail soon, we must find or create a situation in which the treated observations and the untreated observations are similar with respect to all the variables that might affect the outcome other than the treatment variable itself. The best way to accomplish this is by conducting a randomized experiment.
(Llaudet and Imai 2023:34, EMPHASIS ADDED)
Random Assignment
In a randomized experiment, also known as a randomized controlled trial (RCT), researchers decide who receives the treatment based on a random process …
- the treatment group consists of the individuals who received the treatment (the group of observations for which X_{i}=1)
- the control group consists of the individuals who did not receive the treatment (the group of observations for which X_{i}=0)
(Llaudet and Imai 2023:34, EMPHASIS ADDED)
Random Assignment
In Soci—316
Random Assignment
The Class Size Example
students • 50/50 split
Student Composition
Treatment (Small Classes)
Control (Regular Classes)
Balance
Δ percentage points
| Group | Treatment | Control | Δ |
|---|
Random Assignment
When treatment assignment is randomized, the only thing that distinguishes the treatment group from the control group, besides the reception of the treatment, is chance. This means that although the treatment and control groups consist of different individuals, the two groups are comparable to each other, on average, in all respects other than whether or not they received the treatment.
Random treatment assignment makes the treatment and control groups on average identical to each other in all observed and unobserved pre-treatment characteristics. Pre-treatment characteristics are the characteristics of the individuals in a study before the treatment is administered.
(Llaudet and Imai 2023:36, EMPHASIS ADDED)
Random Assignment
RANDOMIZATION OF TREATMENT ASSIGNMENT
By randomly assigning treatment, we ensure that treatment and control groups are, on average, identical to each other in all observed and unobserved pre-treatment characteristics.
Adaptation of summary box in Llaudet and Imai (2023:36).
Random Assignment
With randomization, this holds for two populations, C (X_{i}=0) and T (X_{i}=1)
Image can be retrieved here.
Random Assignment and ATEs
If the treatment and control groups were comparable before the treatment was administered … we can use the factual outcome of one group as an approximation for the counterfactual outcome of the other. In other words, we can assume that the average outcome of the treatment group is a good estimate of the average outcome of the control group, had the control group received the treatment. Similarly, we can assume that the average outcome of the control group is a good estimate of the average outcome of the treatment group, had the treatment group not received the treatment. As a result, we can approximate the average treatment effect by computing the difference in the average outcomes between the treatment and control groups. Since both of these average outcomes are observed, this is an analysis we are able to perform.
(Llaudet and Imai 2023:37, EMPHASIS ADDED)
Random Assignment and ATEs
IF GROUPS WERE COMPARABLE BEFORE
THE TREATMENT WAS ADMINISTERED
THE TREATMENT WAS ADMINISTERED
\widehat{\textit{ATE}}
\;=\;
\overline{Y}_{\text{Treatment Group, }T}
\;-\;
\overline{Y}_{\text{Control Group, } C}
Where
- \widehat{\textit{ATE}} stands for the estimated average treatment effect (the “hat” indicates this is an estimate).
- \overline{Y}_{\text{Treatment Group, } T} is the observed average outcome for the treatment group, and \overline{Y}_{\text{Control Group, } C} is the observed average outcome for the control group.
Adaptation of summary box in Llaudet and Imai (2023:37).
Random Assignment and ATEs
ESTIMATING AVERAGE CAUSAL EFFECTS USING RANDOMIZED EXPERIMENTS AND THE DIFFERENCE-IN-MEANS ESTIMATOR
By using random treatment assignment, we can assume that the treatment and control groups were comparable before the administration of the treatment. As a result, we can rely on the difference-in-means estimator to provide a valid estimate of the average treatment effect.
Adaptation of summary box in Llaudet and Imai (2023:38).
A Class Conversation
The Challenges of Causal Inference
As Llaudet and Imai (2023:38) note, social scientists often run into ethical, logistical, and financial obstacles that ensure that fielding a randomized experiment is challenging—if not impossible.
With this in mind, think about—and refine—your own causal question once again. Can you randomly sort respondents into C (X_{i}=0) and T (X_{i}=1)? How might you go about doing so? What are some challenges you may run into?
0/1000
Copied
See You Thursday
References
Why might class size at Amherst College be correlated with an alum’s income at age 30?
Field of Study is a confounder in this instance.
Llaudet, Elena, and Kōsuke Imai. 2023. Data Analysis for Social Science: A Friendly and Practical Introduction. Princeton (N.J.) Oxford: Princeton University press.
Small, Mario Luis. 2013. “Causal Thinking and Ethnographic Research.” American Journal of Sociology 119(3):597–601. doi: 10.1086/675893.
