Direkt zum Inhalt

Measurement Equivalence in Probability and Nonprobability Online Panels

Referierte Aufsätze Web of Science

Hafsteinn Einarsson, Joseph W. Sakshaug, Alexandru Cernat, Carina Cornesse, Annelies G. Blom

In: International Journal of Market Research 64 (2022), 4, S. 484–505


Nonprobability online panels are commonly used in the social sciences as a fast and inexpensive way of collecting data in contrast to more expensive probability-based panels. Given their ubiquitous use in social science research, a great deal of research is being undertaken to assess the properties of nonprobability panels relative to probability ones. Much of this research focuses on selection bias, however, there is considerably less research assessing the comparability (or equivalence) of measurements collected from respondents in nonprobability and probability panels. This article contributes to addressing this research gap by testing whether measurement equivalence holds between multiple probability and nonprobability online panels in Australia and Germany. Using equivalence testing in the Confirmatory Factor Analysis framework, we assessed measurement equivalence in six multi-item scales (three in each country). We found significant measurement differences between probability and nonprobability panels and within them, even after weighting by demographic variables. These results suggest that combining or comparing multi-item scale data from different sources should be done with caution. We conclude with a discussion of the possible causes of these findings, their implications for survey research, and some guidance for data users.

Carina Cornesse

Head of SOEP Innovation Sample & Social Cohesion Panel in the German Socio-Economic Panel study Department

Keywords: Confirmatory factor analysis, latent variables, multi-item scales, panel vendors, web surveys