Survey Response Behavior as a Proxy for Unobserved Ability: Theory and Evidence

Abstract

An emerging literature is experimenting with using survey response behavior as a proxy for hard-to-measure abilities. We contribute to this literature by formalizing this idea and evaluating its benefits and risks. Using a standard and nationally representative survey from Australia, we demonstrate that the survey item-response rate (SIRR), a straightforward summary measure of response behavior, varies more with cognitive than with noncognitive ability. We evaluate whether SIRR is a useful proxy to reduce ability-related biases in a standard economic application. We show empirically that SIRR, although a weak and imperfect proxy, leads to omitted-variable bias reductions of up to 20%, and performs better than other proxy variables derived from paradata. Deriving the necessary and sufficient conditions for a valid proxy, we show that a strong proxy is neither a necessary nor a sufficient condition to reduce estimation biases. A critical consideration is to which degree the proxy introduces a multicollinearity problem, a finding of general interest. We illustrate the theoretical derivations with an empirical application.