Experimental Research Design Formula Sheet
This sheet is a recall index, not a substitute for the concept notes. Study the interpretation and assumptions in the linked pages, then use the four exam workshops for worked solutions and practice: Foundations Exam Workshop, Regression Exam Workshop, Randomized Experiments Exam Workshop, and Observational Causal Inference Exam Workshop.
Notation convention
Here $z_{\alpha/2}$ and $t_{\alpha/2,df}$ mean positive upper-tail critical values. Equivalently, some books write $z_{1-\alpha/2}$ and $t_{1-\alpha/2,df}$.
1. Probability and sampling distributions
\[\bar X=\frac{1}{n}\sum_{i=1}^nX_i, \qquad s^2=\frac{1}{n-1}\sum_{i=1}^n(X_i-\bar X)^2.\] \[E[X]=\sum_x xP(X=x) \quad\text{or}\quad E[X]=\int_{-\infty}^{\infty}xf(x)\,dx.\] \[\operatorname{Var}(X)=E[X^2]-E[X]^2.\] \[E[aX+b]=aE[X]+b, \qquad \operatorname{Var}(aX+b)=a^2\operatorname{Var}(X).\] \[SE(\bar X)=\frac{\sigma}{\sqrt n}\approx\frac{s}{\sqrt n}.\] \[\frac{\bar X-\mu}{\sigma/\sqrt n}\approx N(0,1).\] \[X\sim\operatorname{Bin}(n,p), \qquad E[X]=np, \qquad \operatorname{Var}(X)=np(1-p).\] \[Z=\frac{X-\mu}{\sigma}\sim N(0,1),\] \[T=\frac{\bar X-\mu}{S/\sqrt n}\sim t_{n-1},\] \[\frac{(n-1)S^2}{\sigma^2}\sim\chi^2_{n-1}.\]See Probability and Sampling Distributions.
2. Hypothesis tests
\[\text{test statistic} = \frac{\text{estimate}-\text{null value}} {\text{standard error}}.\]One mean, known $\sigma$:
\[Z=\frac{\bar X-\mu_0}{\sigma/\sqrt n}.\]One mean, unknown $\sigma$:
\[T=\frac{\bar X-\mu_0}{s/\sqrt n}.\]One proportion:
\[Z=\frac{\hat p-p_0}{\sqrt{p_0(1-p_0)/n}}.\]Two independent means, Welch:
\[T= \frac{(\bar X_1-\bar X_2)-\Delta_0} {\sqrt{s_1^2/n_1+s_2^2/n_2}}.\]Equal-variance pooled test:
\[s_p^2= \frac{(n_1-1)s_1^2+(n_2-1)s_2^2} {n_1+n_2-2},\] \[T=\frac{\bar X_1-\bar X_2} {s_p\sqrt{1/n_1+1/n_2}}.\]Paired test:
\[D_i=X_i-Y_i, \qquad T=\frac{\bar D-\mu_{D,0}}{s_D/\sqrt n}.\]Two proportions under $H_0:p_1=p_2$:
\[\hat p=\frac{x_1+x_2}{n_1+n_2},\] \[Z=\frac{\hat p_1-\hat p_2} {\sqrt{\hat p(1-\hat p)(1/n_1+1/n_2)}}.\]See Hypothesis Tests for Experimental Research.
3. Confidence intervals
\[\bar X\pm z_{\alpha/2}\frac{\sigma}{\sqrt n},\] \[\bar X\pm t_{\alpha/2,n-1}\frac{s}{\sqrt n}.\] \[(\bar X_1-\bar X_2) \pm t^*SE(\bar X_1-\bar X_2).\] \[\hat p\pm z_{\alpha/2} \sqrt{\frac{\hat p(1-\hat p)}{n}}.\]See Confidence Intervals for Experimental Research.
4. ANOVA
\[SST=\sum_i\sum_j(Y_{ij}-\bar Y)^2,\] \[SSB=\sum_i n_i(\bar Y_i-\bar Y)^2,\] \[SSE=\sum_i\sum_j(Y_{ij}-\bar Y_i)^2,\] \[SST=SSB+SSE.\] \[MSB=\frac{SSB}{k-1}, \qquad MSE=\frac{SSE}{N-k}, \qquad F=\frac{MSB}{MSE}.\]See ANOVA for Experimental Research.
5. Simple linear regression
\[Y_i=\beta_0+\beta_1X_i+\varepsilon_i.\] \[\min_{\beta_0,\beta_1} \sum_i(Y_i-\beta_0-\beta_1X_i)^2.\] \[\hat\beta_1= \frac{\sum_i(X_i-\bar X)(Y_i-\bar Y)} {\sum_i(X_i-\bar X)^2}, \qquad \hat\beta_0=\bar Y-\hat\beta_1\bar X.\] \[\hat Y_i=\hat\beta_0+\hat\beta_1X_i, \qquad e_i=Y_i-\hat Y_i.\] \[\hat\sigma^2=\frac{\sum_i e_i^2}{n-2}.\] \[t=\frac{\hat\beta_j-\beta_{j,0}}{SE(\hat\beta_j)}.\] \[R^2=1-\frac{SSE}{SST}.\]See Linear Regression for Experimental Research.
6. Multiple regression
\[Y=X\beta+\varepsilon.\] \[\boxed{\hat\beta=(X^\top X)^{-1}X^\top Y}.\] \[\operatorname{Var}(\hat\beta\mid X)= \sigma^2(X^\top X)^{-1}.\] \[t_j=\frac{\hat\beta_j}{SE(\hat\beta_j)}\]for $H_0:\beta_j=0$. See Linear Regression for Experimental Research.
7. Indicator variables and interactions
\[Y_i=\beta_0+\beta_1D_i+\varepsilon_i, \qquad D_i\in\{0,1\}.\] \[E[Y\mid D=1]-E[Y\mid D=0]=\beta_1.\] \[Y=\beta_0+\beta_1X+\beta_2D+\beta_3XD+\varepsilon.\]See Indicator Variables and Interactions.
8. Binary dependent variable
Linear probability model:
\[P(Y=1\mid X)=X\beta.\]Logistic model:
\[P(Y=1\mid X)=\frac{1}{1+e^{-X\beta}}.\] \[\log\frac{p}{1-p}=X\beta, \qquad OR=e^{\beta_j}.\]See Binary Outcome Models for Experimental Research.
9. Potential outcomes
\[Y_i(1),\qquad Y_i(0),\] \[\tau_i=Y_i(1)-Y_i(0),\] \[\boxed{ATE=E[Y(1)-Y(0)]},\] \[\tau_S=\frac{1}{N}\sum_{i=1}^N[Y_i(1)-Y_i(0)].\] \[\boxed{Y_i^{obs}=W_iY_i(1)+(1-W_i)Y_i(0)}.\] \[W_i=\begin{cases} 1,&\text{treated},\\ 0,&\text{control}. \end{cases}\]See Potential Outcomes Framework.
10. Randomized experiments
\[\boxed{\hat\tau=\bar Y_1-\bar Y_0},\] \[\bar Y_1=\frac{1}{N_1}\sum_{i:W_i=1}Y_i^{obs}, \qquad \bar Y_0=\frac{1}{N_0}\sum_{i:W_i=0}Y_i^{obs}.\]Bernoulli assignment:
\[W_i\sim\operatorname{Bernoulli}(p), \qquad P(W_i=1)=p,\] \[P(W=w)=\prod_{i=1}^N p^{w_i}(1-p)^{1-w_i}.\]Complete randomization:
\[\sum_{i=1}^NW_i=N_1, \qquad \#\mathcal W=\binom{N}{N_1},\] \[P(W=w)=\frac{1}{\binom{N}{N_1}}\]for each allowed assignment. See Randomized Assignment.
11. Neyman inference
\[\hat\tau=\bar Y_1-\bar Y_0.\] \[\operatorname{Var}(\hat\tau)= \frac{S_1^2}{N_1}+ \frac{S_0^2}{N_0}- \frac{S_\tau^2}{N}.\] \[\boxed{ \widehat{\operatorname{Var}}(\hat\tau)= \frac{s_1^2}{N_1}+ \frac{s_0^2}{N_0} }.\] \[SE(\hat\tau)= \sqrt{\frac{s_1^2}{N_1}+\frac{s_0^2}{N_0}}.\] \[\hat\tau\pm z_{\alpha/2}SE(\hat\tau).\]See Neyman Repeated-Sampling Inference.
12. Fisher randomization inference
\[\boxed{H_0:Y_i(1)=Y_i(0)\quad\forall i}.\] \[p=P\left( |T(W,Y)|\ge|T^{obs}| \mid H_0 \right),\]usually with
\[T=\bar Y_1-\bar Y_0.\]See Fisher Randomization Inference.
13. Stratified experiments
\[\hat\tau_k=\bar Y_{1k}-\bar Y_{0k},\] \[\boxed{ \hat\tau=\sum_{k=1}^K\frac{N_k}{N}\hat\tau_k }.\]See Blocked and Paired Randomized Experiments.
14. Paired randomized experiments
\[D_j=Y_{j,T}-Y_{j,C},\] \[\boxed{\hat\tau=\frac{1}{J}\sum_{j=1}^J D_j},\] \[s_D^2=\frac{1}{J-1}\sum_{j=1}^J(D_j-\bar D)^2, \qquad SE(\hat\tau)=\frac{s_D}{\sqrt J}.\]See Blocked and Paired Randomized Experiments.
15. Regression-based treatment effects
\[Y_i=\alpha+\tau W_i+\varepsilon_i \quad\Longrightarrow\quad \boxed{\hat\tau=\bar Y_1-\bar Y_0}.\] \[Y_i=\alpha+\tau W_i+X_i^\top\beta+\varepsilon_i.\] \[Y_i=\alpha+\tau W_i+ X_i^\top\beta+ W_iX_i^\top\gamma+ \varepsilon_i.\]See Regression Adjustment in Experiments.
16. Unconfounded assignment
\[\boxed{(Y(0),Y(1))\perp W\mid X},\] \[\boxed{0<P(W=1\mid X)<1}.\]See Unconfoundedness and Overlap.
17. Propensity score
\[\boxed{e(X)=P(W=1\mid X)}.\] \[\hat e(X)= \frac{1}{1+\exp[-(\hat\beta_0+X^\top\hat\beta)]}.\] \[W\perp X\mid e(X).\]Under unconfoundedness,
\[(Y(0),Y(1))\perp W\mid e(X).\]See Propensity Scores.
18. Propensity weighting
\[\boxed{ \hat\tau_{IPW} = \frac{1}{N}\sum_{i=1}^N \left[ \frac{W_iY_i^{obs}}{\hat e(X_i)} - \frac{(1-W_i)Y_i^{obs}}{1-\hat e(X_i)} \right] }.\]See Inverse Probability Weighting.
19. Matching
\[j(i)=\arg\min_{j:W_j\ne W_i}d(X_i,X_j),\]or
\[j(i)=\arg\min_{j:W_j\ne W_i} |\hat e(X_i)-\hat e(X_j)|.\]For one control matched to each treated unit,
\[\hat\tau_{ATT}=\frac{1}{N_T} \sum_{i:W_i=1}(Y_i^{obs}-Y_{j(i)}^{obs}).\]See Matching for Causal Inference.
Highest-priority recall
\[\boxed{ t\text{-tests},\quad F_{ANOVA},\quad \hat\beta=(X^\top X)^{-1}X^\top Y }\] \[\boxed{ Y^{obs}=WY(1)+(1-W)Y(0),\quad \hat\tau=\bar Y_1-\bar Y_0 }\] \[\boxed{ \operatorname{Var}(\hat\tau),\quad H_0:Y_i(1)=Y_i(0)\ \forall i }\] \[\boxed{ e(X)=P(W=1\mid X),\quad (Y(0),Y(1))\perp W\mid X }.\]Fast selection guide
| Question structure | Main formula or method |
|---|---|
| One numerical mean | One-sample $z$ or $t$ |
| Two independent numerical means | Welch $t$ |
| Same units measured twice | Paired differences |
| More than two group means | ANOVA $F$ |
| Randomized treatment-control effect | Difference in means plus design-based inference |
| Sharp no-effect-for-any-unit null | Fisher randomization test |
| Randomization within strata or pairs | Weighted stratum effects or pair differences |
| Observational treatment with measured confounders | Adjustment under unconfoundedness and overlap |
| Inverse-probability pseudo-population | IPW |
| Comparable treated and control units | Matching with an explicit estimand |
Do not mix these up
| Similar-looking ideas | Distinction |
|---|---|
| Known versus unknown population variance | Use $z$ with known $\sigma$ and $t$ with estimated $s$ for the classical one-mean problem. |
| Proportion test versus proportion interval | The test of $p=p_0$ uses $p_0$ in its null standard error; the usual Wald interval uses $\hat p$. |
| Welch versus pooled two-sample $t$ | Welch allows unequal variances; the pooled statistic assumes a common population variance. |
| Independent versus paired observations | Independent groups use two group variances; paired data reduce to one sample of differences $D_i$. |
| $SST$, $SSB$, and $SSE$ in ANOVA | $SST$ is total, $SSB$ is between groups, and $SSE$ is within groups. |
| $SSE$ and $SSR$ across textbooks | Many texts use $SSE$ for residual error and $SSR$ for explained regression variation, while some econometrics texts use $SSR$ for the sum of squared residuals. Define the symbols before calculating $R^2$ or an $F$ statistic. |
| Fisher and Neyman nulls | Fisher’s sharp null says every $\tau_i=0$; a zero average effect allows heterogeneous effects to cancel. |
| Sampling and assignment distributions | Classical inference often imagines repeated samples; design-based experimental inference holds units fixed and repeats treatment assignment. |
| ATE and ATT | The IPW formula above targets the ATE; matching one control to every treated unit naturally targets the ATT. |
| Randomized pairs and observational matching | Matched-pair experiments randomize treatment inside pairs; observational matching still requires unconfoundedness. |