This tab shows why the estimand has to come before the estimator. The same DiD design can answer different economic questions depending on whether the target is the typical unit, the population total, or a level effect. With heavy-tailed outcomes, these estimands can differ sharply, and sometimes even imply effects with opposite signs. Use the sliders below to change the data-generating process and see how the true estimands and the estimated coefficients move.
The simulation creates a matched treated–control panel that simplifies the setting for educational purposes, isolating three ingredients: outcome concentration (the share of outcomes in the top decile), heterogeneous treatment effects (separate effects for the viral head and the long tail), and treatment-induced changes in variance (the proportional change in treated-unit volatility). Treated and control units have the same pre-period baseline levels by construction, so differences across estimators come from the estimand they target and how they weight the outcome distribution. For the case where treated and control units start at different levels, see the Levels-OLS bias map tab →
(pre-treatment distribution)
Each estimator targets a different estimand and applies a different implicit or explicit weighting scheme. Log(1+Y) OLS weights observations equally, so the many low-volume units dominate the estimate. Weighted log OLS uses explicit pre-period outcome shares. PPML weights through the predicted mean and is the natural estimator for the population-total percent effect. Levels OLS targets an effect in units of the outcome. When treatment also changes outcome variance, log-transformed specifications can introduce a bias and even reverse sign. Setting Δ variance to zero isolates the role of weighting and estimand choice.
Tab 1 assumed matched treated and control units, so baseline level differences were zero by construction. This tab asks what happens when the matched-baseline assumption is relaxed. We keep the Tab 1 DGP fixed, including the treatment effects, variance shift, outcome concentration, and panel structure. We then add two choices, defined by the sliders below: a treated-control baseline gap (b) and common proportional growth (g). Each maps to a factor shown below, and the two multiply to give the levels-OLS bias under the multiplicative trend assumed here. Under an additive trend, the bias would instead appear in PPML and log-based estimators.
Good news: with matched baseline levels between treated and control outcomes, this bias channel largely disappears; there is no level gap for proportional growth to amplify, so researchers do not need to worry about additive vs. multiplicative trend structure for credible inference.
Move the b and g sliders (or drag the ✕) to see how the bias changes.
Under common proportional growth g, absolute changes scale with baseline levels: a treated unit (1+b) times larger mechanically adds (1+b) times more under the no-treatment counterfactual, and a levels DiD attributes the difference to treatment. With balanced windows the bias is exactly [b/(1+b)] × [(Ḡpost − Ḡpre)/Ḡpost], with Ḡ the window-average growth factor; in this 10+10 design the trend-scale factor is 1 − (1+g)−10. Either parameter at zero closes the channel: matching on baseline levels shrinks b toward zero, and no common growth means no trend-scale factor. The bias is separate from the treatment effect and adds to the scaled levels estimate. Proportional estimators (PPML, log specifications) impose proportional counterfactual trends and carry no bias from this channel. See Roth & Sant’Anna (2023) for more details.
Synthetic difference-in-differences (Arkhangelsky et al., 2021) reweights control units (ω) and pre-periods (λ) so the control group matches the treated group’s pre-trend, then estimates the effect from a weighted two-way fixed-effects DiD. The reweighting relaxes parallel trends and absorbs level differences between the groups. We apply it to the same simulated panel as Tab 1 and Tab 2, with the treated units as the treated group and the matched control units as the donor pool. Synthetic DiD can be run on either scale: in levels it targets the population-total effect, in logs the typical unit. Levels is the more common default, though Arkhangelsky et al. themselves apply it to log GDP. In levels its period weights reduce the baseline-gap bias of Tab 2 but do not remove it, and with heavy-tailed outcomes the levels estimate is the least precise of the estimators here.
The sliders below set the outcome concentration, the baseline gap and growth that generate the levels bias, and the treatment-induced variance change that drives the log bias; every other feature of the data-generating process follows Tab 1.
A confidence interval needs a standard error, and for synthetic DiD the choice matters: the estimator reweights the data before it estimates, so a standard error that treats those weights as fixed is not estimating the right object. The convention in applied work is to use one of the resampling estimators that refit the weights; the synthdid package defaults to the jackknife, which is what the SDID intervals above use. Four estimators are worth comparing:
- ω,λ fixed (clustered). The cluster-robust standard error computed once with the ω and λ weights held fixed (the “ω,λ fixed” column below). This is the baseline to be wary of for SDID: it ignores that the weights were themselves estimated, so it is not the right object. Shown for comparison, not as the default.
- Jackknife. Drop groups of units and re-run the whole procedure — refitting the weights — on each subset, then read the variance off the leave-out estimates. The package default, and the SDID interval reported in the chart above.
- Bootstrap. Resample the matched pairs with replacement, re-run on each sample, and take the spread of the estimates.
- Placebo. Reassign treatment among the control units to trace out a null distribution — built for a small treated group against a larger donor pool (Arkhangelsky et al., 2021). This design has equally many treated and control units, so to run it at the full sample we mint extra control-only units — possible only because we know the data-generating process — and draw the placebo assignments from that enlarged donor pool.
The two charts below plot the interval under all four methods — for synthetic DiD in levels and in logs — against the empirical spread of the estimate over fresh draws of the panel, the quantity a standard error is meant to recover. Both share the y-axis of the chart above.
Synthetic DiD improves on a plain levels DiD, but it remains a levels estimator: under the multiplicative data-generating process of Tab 1 and Tab 2 it carries a residual baseline-gap bias and, with heavy-tailed outcomes, a wider confidence interval than PPML. PPML reaches the same population-total effect more precisely. The estimand is set by the choice of outcome scale, not by the estimator; the typical-unit effect is the subject of Tab 1.
The applet generates the data; the code estimates the specifications. The buttons below export the simulated panel for the current slider settings as a CSV file. The code below (in Python, R, and Julia) reads that file, runs the specifications from Tabs 1 and 2 — unchanged from v5. A separate, clearly-fenced block at the end of each script adds the Tab 3 synthetic-DiD estimator, so it is easy to see exactly what is new. The zip also includes a reference panel for the default settings.
did_lab_dgp.R (base R — generates the panel CSV from the slider settings + a seed)
did_lab_replication.py (requires numpy, pandas, scipy, pyfixest, synthdid — click to view)
did_lab_replication.R (requires fixest, synthdid — click to view)
did_lab_replication.jl (requires DataFrames, FixedEffectModels, GLFixedEffectModels — click to view)
DGP and specification details
Balanced matched-pair panel. Each treated unit i has baseline b0i(1+b); its matched control has b0i, where b is the Tab 2 baseline-gap slider (b = 0 on Tab 1). The b0i sit at the midpoint quantiles of lognormal(0, σ²pop), and σpop is solved numerically on that quantile grid so the realized top-decile share of the total equals the “% of outcome in top decile” slider. Panel: 10 pre + 10 post periods. Outcomes: Yit = b0i·(1+g)t·(1+θi)Dit·εit·100, rounded to integer counts, with g the Tab 2 common-growth slider (g = 0 on Tab 1). θi equals the head slider for the top size decile and the tail slider for the rest. εit is mean-one multiplicative lognormal noise, eσZ−σ²/2, composed of a pair-common component (cv 0.12, shared by the two units of a pair in each period) and a unit-idiosyncratic component. Baseline total cv: σi = 0.3·(b0i/b0,med)−0.2, clipped to [0.12, 0.60]. Treatment multiplies the idiosyncratic component of treated-unit volatility by √(1+Δvar) in post periods, for all treated units; the pair-common component is unchanged. True estimands: typical-unit = equal-weighted mean of θi; population-total = baseline-weighted mean of θi. Specifications: every estimator uses unit and period fixed effects with a single Treat×Post regressor — OLS in levels, OLS on log(1+Y), weighted OLS on log(1+Y) with pre-period mean outcome weights, and Poisson pseudo-maximum-likelihood (log link, reported as eδ−1). Standard errors are CRV1, clustered by matched pair, with t critical values on pairs−1 degrees of freedom; two-way clustering by unit and pair is identical here because units are nested within pairs. The levels coefficient is reported as a percent of the treated pre-period mean multiplied by the control post/pre growth ratio, with a delta-method confidence interval. The Tab 2 bias surface is the exact probability limit [b/(1+b)] · [(Ḡpost−Ḡpre)/Ḡpost], with Ḡ the within-window mean of (1+g)t.