Working papers

• A Design-Based Riesz Representation Framework for Randomized Experiments. Abstract

  With Christopher Harshaw and Yitan Wang.
  We describe a new design-based framework for drawing causal inference in randomized experiments. Causal effects in the framework are defined as linear functionals evaluated at potential outcome functions. Knowledge and assumptions about the potential outcome functions are encoded as function spaces. This makes the framework expressive, allowing experimenters to formulate and investigate a wide range of causal questions. We describe a class of estimators for estimands defined using the framework and investigate their properties. The construction of the estimators is based on the Riesz representation theorem. We provide necessary and sufficient conditions for unbiasedness and consistency. Finally, we provide conditions under which the estimators are asymptotically normal, and describe a conservative variance estimator to facilitate the construction of confidence intervals for the estimands.

• Optimized variance estimation under interference and complex experimental designs. Abstract

  With Christopher Harshaw and Joel Middleton.
  Unbiased and consistent variance estimators generally do not exist for design-based treatment effect estimators because experimenters never observe more than one potential outcome for any unit. The problem is exacerbated by interference and complex experimental designs. In this paper, we consider variance estimation for linear treatment effect estimators under interference and arbitrary experimental designs. Experimenters must accept conservative estimators in this setting, but they can strive to minimize the conservativeness. We show that this task can be interpreted as an optimization problem in which one aims to find the lowest estimable upper bound of the true variance given one's risk preference and knowledge of the potential outcomes. We characterize the set of admissible bounds in the class of quadratic forms, and we demonstrate that the optimization problem is a convex program for many natural objectives. This allows experimenters to construct less conservative variance estimators, making inferences about treatment effects more informative. The resulting estimators are guaranteed to be conservative regardless of whether the background knowledge used to construct the bound is correct, but the estimators are less conservative if the knowledge is reasonably accurate.

• Nonparametric identification is not enough, but randomized controlled trials are. Abstract

  With P.M. Aronow, James Robins, Theo Saarinen and Jasjeet Sekhon.
  We argue that randomized controlled trials (RCTs) are special even among settings where average treatment effects are identified by a nonparametric unconfoundedness assumption. This claim follows from two results of Robins and Ritov (1997): (1) with at least one continuous covariate control, no estimator of the average treatment effect exists which is uniformly consistent without further assumptions, (2) knowledge of the propensity score yields a uniformly consistent estimator and honest confidence intervals that shrink at parametric rates with increasing sample size, regardless of how complicated the propensity score function is. We emphasize the latter point, and note that successfully-conducted RCTs provide knowledge of the propensity score to the researcher. We discuss modern developments in covariate adjustment for RCTs, noting that statistical models and machine learning methods can be used to improve efficiency while preserving finite sample unbiasedness. We conclude that statistical inference has the potential to be fundamentally more difficult in observational settings than it is in RCTs, even when all confounders are measured.

Recent publications

• Balancing Covariates in Randomized Experiments with the Gram–Schmidt Walk Design. Abstract

  With Christopher Harshaw, Daniel Spielman and Peng Zhang.
  Journal of the American Statistical Association (2024), in print.
  The design of experiments involves a compromise between covariate balance and robustness. This paper provides a formalization of this trade-off and describes an experimental design that allows experimenters to navigate it. The design is specified by a robustness parameter that bounds the worst-case mean squared error of an estimator of the average treatment effect. Subject to the experimenter’s desired level of robustness, the design aims to simultaneously balance all linear functions of potentially many covariates. Less robustness allows for more balance. We show that the mean squared error of the estimator is bounded in finite samples by the minimum of the loss function of an implicit ridge regression of the potential outcomes on the covariates. Asymptotically, the design perfectly balances all linear functions of a growing number of covariates with a diminishing reduction in robustness, effectively allowing experimenters to escape the compromise between balance and robustness in large samples. Finally, we describe conditions that ensure asymptotic normality and provide a conservative variance estimator, which facilitate the construction of asymptotically valid confidence intervals.

• Causal inference with misspecified exposure mappings: separating definitions and assumptions. Abstract

  Biometrika (2024), 111(1), 1–15.
  Exposure mappings facilitate investigations of complex causal effects when units interact in experiments. Current methods require experimenters to use the same exposure mappings to define the effect of interest and to impose assumptions on the interference structure. However, the two roles rarely coincide in practice, and experimenters are forced to make the often questionable assumption that their exposures are correctly specified. This paper argues that the two roles exposure mappings currently serve can, and typically should, be separated, so that exposures are used to define effects without necessarily assuming that they are capturing the complete causal structure in the experiment. The paper shows that this approach is practically viable by providing conditions under which exposure effects can be precisely estimated when the exposures are misspecified. Some important questions remain open.

• Design and analysis of bipartite experiments under a linear exposure-response model. Abstract

  With Christopher Harshaw, David Eisenstat, Vahab Mirrokni and Jean Pouget-Abadie.
  Electronic Journal of Statistics (2023), 17(1), 464–518.
  A bipartite experiment consists of one set of units being assigned treatments and another set of units for which we measure outcomes. The two sets of units are connected by a bipartite graph, governing how the treated units can affect the outcome units. In this paper, we consider estimation of the average total treatment effect in the bipartite experimental framework under a linear exposure-response model. We introduce the Exposure Reweighted Linear (ERL) estimator, and show that the estimator is unbiased, consistent and asymptotically normal, provided that the bipartite graph is sufficiently sparse. To facilitate inference, we introduce an unbiased and consistent estimator of the variance of the ERL point estimator. Finally, we introduce a cluster-based design, Exposure-Design, that uses heuristics to increase the precision of the ERL estimator by realizing a desirable exposure distribution.

Software

• GSWDesign.jl

  Julia package with a fast implementation of the Gram-Schmidt Walk for balancing covariates in randomized experiments (also R wrapper).

• distances

  R package with tools for distance metrics.

• quickmatch

  Quick Generalized Full Matching in R.

• quickblock

  Quick Threshold Blocking in R.

• scclust

  C library for size-constrained clustering.