Comment: The Essential Role of Policy Evaluation for the 2020 Census Disclosure Avoidance System
We’re excited to announce our forthcoming article discussing boyd and Sarathy (2022).
We’re excited to announce a new paper, Comment: The Essential Role of Policy Evaluation for the 2020 Census Disclosure Avoidance System, now forthcoming at the Harvard Data Science Review. This article responds to boyd and Sarathy (2022). This builds on our research into the impact of differential privacy on redistricting and advances our responses to some common questions.
The abstract is listed below:
In “Differential Perspectives: Epistemic Disconnects Surrounding the US Census Bureau’s Use of Differential Privacy,” boyd and Sarathy argue that empirical evaluations of the Census Disclosure Avoidance System (DAS), including our published analysis, failed to recognize how the benchmark data against which the 2020 DAS was evaluated is never a ground truth of population counts. In this commentary, we explain why policy evaluation, which was the main goal of our analysis, is still meaningful without access to a perfect ground truth. We also point out that our evaluation leveraged features specific to the decennial Census and redistricting data, such as block-level population invariance under swapping and voter file racial identification, better approximating a comparison with the ground truth. Lastly, we show that accurate statistical predictions of individual race based on the Bayesian Improved Surname Geocoding, while not a violation of differential privacy, substantially increases the disclosure risk of private information the Census Bureau sought to protect. We conclude by arguing that policy makers must confront a key trade-off between data utility and privacy protection, and an epistemic disconnect alone is insufficient to explain disagreements between policy choices.
For those interested in the full commentary, a preprint is available here.