List Experiments

Citation: Aronow, Peter M., Alexander Coppock, Forrest W. Crawford, and Donald P. Green. 2015. “Combining List Experiment and Direct Question Estimates of Sensitive Behavior Prevalence.” Journal of Survey Statistics and Methodology. 3(1): 43-66.

Abstract: Survey respondents may give untruthful answers to sensitive questions when asked directly. In recent years, researchers have turned to the list experiment (also known as the item count technique) to overcome this difficulty. While list experiments are arguably less prone to bias than direct questioning, list experiments are also more susceptible to sam- pling variability. We show that researchers need not abandon direct questioning altogether in order to gain the advantages of list experimen- tation. We develop a nonparametric estimator of the prevalence of sensi- tive behaviors that combines list experimentation and direct questioning. We prove that this estimator is asymptotically more efficient than the standard difference-in-means estimator, and we provide a basis for infer- ence using Wald-type confidence intervals. Additionally, leveraging information from the direct questioning, we derive two nonparametric placebo tests for assessing identifying assumptions underlying list experiments. We demonstrate the effectiveness of our combined estimator and placebo tests with an original survey experiment.

Link to article here.

Link to pdf here.

Link to online appendix here.

Link to replication archive here.

Figure 1 from paper, showing the gain in precision from using the combined estimator across 5 list experiments.