Does the mode of data collection matter in demographic research? Insights from a GGP mode experiment study

Andrej Kveder, Netherlands Interdisciplinary Demographic Institute (NIDI)
Joop Hox, Utrecht University
Katja Lozar Manfreda, University of Ljubljana
Gregor Petrič, University of Ljubljana
Jernej Berzelak, University of Ljubljana
Rok Platinovsek, University of Ljubljana

Current survey taking climate is challenging for researchers in demography and social sciences in general. Response to surveys has been in constant decline in the past decades making the collection of new data increasingly costly. Developments in survey methodology and especially the feasibility of web surveys offer new opportunities for social science research. However, different data collection modes can yield substantively different results. The aim of this paper is to investigate to what extent mode (CAPI, CATI, WEB) differences exist in a large-scale demographic survey, and explore the feasibility of web implementation of a questionnaire that is both long and complex (with retrospective histories, prospective elements and social network measures). Although considerable amount of research has been accomplished studying mode effects (de Leeuw, 1992, 2005), there is no scientific evidence relating to large-scale demographic surveys such as the Generations and Gender Survey, the survey analysed in this paper. The design of the study employs a randomized design into three selected modes of data collection. Each individual stayed with the assigned mode and was not offered a switch in order to keep the modes “ clean”. The analysis focuses on the comparison of modes in terms of data quality (item nonresponse) and measurement equivalence (distribution comparisons). The comparative analysis of modes aggregates questions using a customized question typology based on general characteristics of questions as well as linguistic constructs that tap into respondents’ cognitive processing. The use of such a typology will provide answers as to which types of questions are sensitive to different mode effects. The results of the proposed paper will provide insights into the possibility of conducting large-scale demographic surveys using web mode of data collection, contribute to the discourse on the quality of demographic data and open avenues for even more innovative demographic research.