Medicaid Expansion, and the Limits of a QALY
A staggered difference-in-differences study of ACA Medicaid expansion on coverage, access, and health.
Between 2014 and 2019, states adopted the ACA Medicaid expansion at different times, and a stubborn handful never adopted it at all. That staggered rollout is about as close to a natural experiment as health policy gets: a known treatment, a known eligibility threshold (138% of the federal poverty line), and decent annual survey data on both sides. I ask three questions, in order. Did expansion cover more eligible adults? Did it pull down the cost wall between them and a doctor? And did their health improve enough to call the program cost-effective in the currency health economists trade in, the cost per quality-adjusted life year?
Why you cannot just run a regression with state and year dummies
The instinct, when units get treated at different times, is the two-way fixed effects regression: outcome on a treatment dummy, with state and year fixed effects mopping up the rest. That was the default specification for a long time. It also breaks whenever treatment timing varies, and the reason took the profession years to corner.
The trouble is what TWFE picks as a control group. When Arkansas expands in 2014 and Louisiana expands in 2016, the regression treats already-expanded Arkansas as a valid comparison for newly-expanded Louisiana. Arkansas is not a clean counterfactual. It already carries its own treatment effect, maybe one still working its way through, and that contaminated comparison earns a positive weight in the average. Goodman-Bacon (2021) worked out the arithmetic: the estimator decomposes into a weighted sum of every possible two-by-two comparison, the forbidden already-treated-as-control ones included, and once treatment effects differ across cohorts those weights can turn negative.
I use Callaway and Sant’Anna’s group-time estimator instead. It compares a cohort that just got treated only against units not yet treated, or never treated, then stitches those clean pieces into an overall effect and an event-study path. TWFE still shows up in the dashboard, as a foil. Put it next to the Goodman-Bacon decomposition and you can see exactly where its bias comes from: on this data, about 13% of the TWFE weight rides on forbidden comparisons, and the TWFE coverage estimate shrinks against the clean one by roughly a third.
Inference clusters on the state, since that is the level the policy varies at, and with around 50 clusters the textbook cluster-robust standard error runs too small. I correct it with a wild cluster bootstrap, the Rademacher variant, which earns its keep when you have few enough clusters to worry but too many to enumerate. I built it from per-state influence functions rather than reaching for a package, so you can audit the few-cluster fix line by line.
Who is actually in the estimand
This matters more than it sounds. The policy targets non-elderly low-income adults. Estimate on all adults and you dilute the effect toward nothing, because most adults were never eligible and never moved.
BRFSS gives you no clean continuous income, so eligibility runs through a proxy: ages 19 to 64, paired with the income bracket that sits at or below roughly 138% of poverty for a typical low-income household. The proxy is rough. The income coding also drifts across survey years, from the old eight-bracket INCOME2 to a later grouped version, and the cleaning layer has to reconcile the two before anything downstream can run. The robustness suite leans on all of this: tighten the cutoff, loosen it, swap the control group, drop the 2014 cohort, and see whether the headline moves. The dilution check is the one I find most convincing. Restrict the sample to the eligible and the coverage effect is large. Open it up to all adults and it nearly disappears. That collapse is what a real, targeted effect should do.
What the coverage numbers say
Among eligible low-income adults, expansion cut the share with no insurance by about 9.4 percentage points. The event study sits flat before adoption and steps down after, the pre-trend test does not reject, and the estimate survives every robustness cut. Of the three questions, this is the one I would defend without hedging.
The “could not see a doctor because of cost” barrier moves too, down about 3.6 points, in the direction you would expect once coverage lowers the price of care at the point of use.
Then it gets complicated.
Where it stops being causal
The cost-barrier and self-rated-health outcomes both fail the pre-trend test. Their event studies are not flat before adoption. Expanding and non-expanding states were already pulling apart on these measures before anyone expanded anything. Once the pre-trend rejects, the parallel-trends assumption the whole design leans on stops holding for that outcome, and the placebo test, which backdates a fake adoption by a few years, lights up when it should stay dark. The diagnostics are doing their job. They tell me I cannot separate the policy’s effect on these outcomes from movements that predate it.
That is not the same as saying expansion did not improve health.
“Medicaid expansion improves the health of the people it covers” is a falsifiable and therefore scientific claim. The catch is that self-rated general health in BRFSS, over this window, against these pre-existing trends, is not the instrument that can test it. Self-rated health is coarse and slow to shift. It is exactly the sort of measure where slow state-level trends swamp a real but modest treatment effect. To test the claim properly you would want claims data, mortality, condition-specific outcomes, a longer post-period. I do not have those. So I report the coverage and access effects, mark the health outcome as not identified on this data, and refuse to launder a contaminated estimate into a clean-looking number.
The health-economics overlay, and the absent ICER
The standard HEOR pipeline takes a health gain, converts it to quality-adjusted life years with a utility weight, divides incremental cost by the QALY gain, and reports a cost per QALY against the usual willingness-to-pay thresholds of 50, 100, and 150 thousand dollars. The cost side holds up: MEPS expenditure data puts the incremental annual medical spend of covering a low-income adult near 6,200 dollars, with the out-of-pocket share falling about forty points as the payer mix shifts.
The QALY side does not hold up. It rests entirely on the self-rated-health effect, and that effect failed identification one section ago.
So the pipeline does something unglamorous. It holds the ICER to the same bar as everything else: the health effect has to clear zero and pass the pre-trend diagnostic before any cost-per-QALY gets computed. It clears neither. The ICER returns “not demonstrated,” with the reason printed in plain text, and the probabilistic sensitivity analysis backs that up. Simulate over the utility weights, the cost, and the uptake together and the probability of being cost-effective at 100 thousand per QALY sits near zero. Not because expansion costs too much for what it delivers, but because the benefit the whole calculation hangs on was never measured cleanly.
What expansion clearly buys, on this data, is coverage and access. Whether it buys health, and at what price, this dataset cannot say.
Data and reproducibility
The causal estimates run on real CDC BRFSS annual files, 2011 through 2019, roughly four million respondents collapsed to a state-by-year panel of survey-weighted means. The cost parameters come from the AHRQ MEPS full-year consolidated file. Treatment timing comes from the KFF expansion-decision tracker. The utility weights are three sourced crosswalks, a CDC HRQOL mapping, the US EQ-5D-3L value set, and SF-6D, and I run all three through the sensitivity analysis rather than betting on one.
A single seed drives everything, the wild bootstrap and the Monte Carlo included, so a clean run reproduces the same JSON to the digit. When the raw survey files are absent, the pipeline falls back to a calibrated synthetic panel with a known planted effect, stamps every output as synthetic, and flags it with a banner, so nobody mistakes a demo for a finding. The headline estimator ships as both a Callaway-Sant’Anna implementation in R, for the reference path, and a tested pure-Python version that returns the identical schema. The Python version is what ran here.
Limitations
The eligibility proxy is the soft underbelly. BRFSS income brackets are a blunt instrument for a 138%-of-poverty line, and the household-size adjustments are approximate. The robustness runs bound that sensitivity without dissolving it.
The pre-trend failures on the access and health outcomes are real, and I report them as such. They cap what this design can claim. No amount of extra specification search fixes that. It only buries it.
The budget-impact figure is stylised. MEPS is national, so the per-adult cost magnitude is a national average laid over an eligible-population count, not a state-matched calculation, and you should read it as an order of magnitude rather than a forecast.
The coverage effect itself, clean as it is, is an intention-to-treat-flavoured average over the eligible population as proxied. It measures the effect of living in an expansion state while eligible, not the effect of enrollment on a complier, which is a different and harder parameter to pin down.