Observational Causal Inference Exam Workshop
Observational questions should begin with a causal diagram or assignment story, not a propensity model. Decide which pretreatment variables are sufficient to block confounding paths, then assess overlap, balance, and the target population.
Worked problem 1: confounding and standardization
An observational training study has two prior-skill strata. Half of the target population is high skill and half is low skill. Within each stratum, the observed mean outcomes are:
| Prior skill | $E[Y\mid W=1,X]$ | $E[Y\mid W=0,X]$ | $e(X)$ |
|---|---|---|---|
| High | 70 | 62 | 0.80 |
| Low | 55 | 50 | 0.20 |
Because 80% of treated units are high skill while only 20% of controls are high skill, the crude observed contrast is
\[[0.8(70)+0.2(55)]-[0.2(62)+0.8(50)]=67-52.4=14.6.\]Under conditional unconfoundedness and overlap, standardize both treatment conditions to the target covariate distribution:
\[E[Y(1)]=0.5(70)+0.5(55)=62.5,\] \[E[Y(0)]=0.5(62)+0.5(50)=56.\]Therefore,
\[ATE=62.5-56=6.5.\]The crude gap combines a treatment contrast with a large composition difference. Standardization removes the part explained by the measured prior-skill stratum, conditional on the identification assumptions.
Worked problem 2: what inverse weights create
For a treated high-skill unit, the ATE weight is $1/0.80=1.25$. For a treated low-skill unit, it is $1/0.20=5$. Control weights reverse the pattern:
\[w_i= \begin{cases} 1/e(X_i), & W_i=1,\\ 1/[1-e(X_i)], & W_i=0. \end{cases}\]Low-skill treated units and high-skill controls are rare relative to their covariate strata, so each receives more weight. The weighted pseudo-population represents both skill strata in both treatment conditions. This explains the estimator; it also explains why estimated propensities near zero or one cause instability.
If a high-skill control has $\hat e(X)=0.98$, its control weight is
\[\frac{1}{1-0.98}=50.\]One observation can then dominate the estimate. Report the propensity distribution, effective sample size, balance after weighting, and any prespecified trimming or overlap-weight strategy.
Worked problem 3: matching estimand and reuse
Suppose three treated units have propensity scores $(0.22,0.51,0.79)$ and four controls have scores $(0.18,0.47,0.55,0.81)$, with outcomes $(12,18,25)$ for treated units and $(9,13,17,20)$ for controls.
One-to-one nearest-neighbor matching without replacement pairs
\[0.22\leftrightarrow0.18, \qquad 0.51\leftrightarrow0.47, \qquad 0.79\leftrightarrow0.81.\]The matched ATT estimate is
\[\widehat{ATT}=\frac{(12-9)+(18-13)+(25-20)}{3}=4.33.\]The unmatched control at 0.55 is outside this particular matched comparison. Matching changes the analyzed population, and a no-replacement rule can force later units into worse matches. The answer should state the estimand, distance, replacement rule, caliper, and balance diagnostics.
Worked problem 4: variable selection by causal role
A study estimates the effect of job training on later earnings. Available variables are age, prior earnings, motivation measured before enrollment, training completion, and earnings six months after training.
- Age, prior earnings, and pretreatment motivation may confound selection and outcome, so they are candidates for the assignment model.
- Training completion occurs after assignment and can mediate the program effect. Conditioning on it changes the estimand and may induce bias.
- Six-month earnings are the outcome and must not enter a propensity model intended to reproduce pretreatment assignment.
Predicting treatment well is not the objective. The objective is covariate balance for a defensible causal comparison.
Exercises
- For the two-stratum table, derive the ATT when 80% of treated units are high skill. Compare it with the ATE and explain why they differ.
- Calculate ATE weights for propensities $0.05$, $0.25$, $0.50$, $0.75$, and $0.95$ under both observed treatment states. Identify the unstable cases.
- Repeat the matching problem with a caliper of 0.03. State which treated units remain unmatched and how the target population changes.
- A propensity model achieves an AUC of 0.91 but leaves a standardized mean difference of 0.28 for prior outcome after weighting. Evaluate the design.
- Draw a causal diagram in which conditioning on a post-treatment variable opens bias. Explain why adding every available variable is unsafe.
- State which parts of unconfoundedness and overlap can be assessed from observed data and which cannot.
Solution checks
- $ATT=0.8(70-62)+0.2(55-50)=7.4$. It weights effects by the treated covariate distribution rather than the target population distribution.
- The largest weights are 20 for a treated unit at 0.05 and for a control at 0.95; both signal weak overlap.
- Only $0.79\leftrightarrow0.81$ lies within 0.03; the estimate then applies to a narrow matched subset.
- Treatment prediction does not establish balance. A standardized difference of 0.28 is substantial residual imbalance, so revise the design before estimating effects.
- A mediator or collider measured after treatment can block part of the effect or open a noncausal path.
- Overlap and observed-covariate balance are diagnosable. Absence of unmeasured confounding is not testable from the observed treatment-outcome data alone.