Robustness Value Cinelli and Hazlett sensitivity to an omitted variable

A regression coefficient is causal only if nothing that drives both the treatment and the outcome was left out. This page takes three numbers from a regression table and says how strong a left-out variable would have to be to bring the estimate to zero, or to make it statistically insignificant.

From the regression table

Observations minus the number of coefficients estimated, the intercept included. The example is the effect of direct exposure to violence on support for peace in Darfur, from Hazlett (2020).

A covariate to compare against (optional)

The example values give the bounds the paper reports for a confounder as strong as being female: 0.9% with the treatment and 12.5% with the outcome. Leave the t-statistics empty to skip this part.

How fragile is the estimate

If the confounder were like the covariate

Adjusted t-statistic for a confounder of a given strength

Rows: the share of the treatment's residual variation the confounder explains. Columns: the share of the outcome's residual variation it explains. Each cell is the t-statistic that would remain, assuming the confounder pulls the estimate toward zero.

Method and limits

Write f = t ÷ √df for the estimate. The partial R² of the treatment with the outcome is f² ÷ (1 + f²). The robustness value for a share q is RV = ½ (√(fq⁴ + 4 fq²) − fq²), with fq = q |f|. A confounder that explains RV of the residual variation in both the treatment and the outcome removes the share q of the estimate. One that explains less than RV in both cannot.

The second robustness value asks for less: only that the confidence interval come to include the target. It replaces fq with fq minus the critical t-value at df − 1, divided by √(df − 1). When the estimate is very precise, with fq above the reciprocal of that critical ratio, the bound that constrains only the confounder's association with the treatment is tighter, and the value is (fq² − f*²) ÷ (1 + fq²).

For a confounder with strengths R²D and R²Y, the bias is at most se × √df × √(R²Y R²D ÷ (1 − R²D)), and the standard error becomes se × √((1 − R²Y) ÷ (1 − R²D)) × √(df ÷ (df − 1)). The comparison with a covariate turns "k times as strong as this covariate" into those two strengths with the bounds in section 4.4 of the paper.

The robustness value is a description of the estimate. It does not say whether such a confounder exists. That is an argument about the setting, and the covariate comparison is only as persuasive as the claim that the left-out variable is no stronger than the one named. The method covers one linear regression coefficient with conventional standard errors. With clustered or robust standard errors the page's answers use the t-statistic you enter and the degrees of freedom you enter, and no adjustment has been derived here for the number of clusters. It does not cover instrumental variables, logit or probit, or several confounders that each fall below the threshold and together exceed it, although the bounds do hold for a set of confounders taken as one.

Sources: Carlos Cinelli and Chad Hazlett, "Making sense of sensitivity: extending omitted variable bias", Journal of the Royal Statistical Society: Series B 82(1), 2020, pages 39 to 67. Chad Hazlett, "Angry or weary? How violence impacts attitudes toward peace among Darfurian refugees", Journal of Conflict Resolution 64(5), 2020.

Need help setting out your story, what sets your work apart, the context, the return on investment and the projections? Pinpoint Ventures can help. Write to hello@pinpointventures.in or message on WhatsApp.