Practical strategies for minimizing sampling and non-sampling errors in telephone surveys: a case study using the “Sample Survey on Births”

Cinzia Castagnaro, Istituto Nazionale di Statistica (ISTAT)
Antonella Guarneri, Istituto Nazionale di Statistica (ISTAT)
Sabrina Prati, Istituto Nazionale di Statistica (ISTAT)
Francesca Rinesi, Istituto Nazionale di Statistica (ISTAT)

It is well-known that Italy is one of the Countries with the lowest fertility level in Europe, therefore surveys devoted to this topic are particularly relevant. Starting from October 2011, a new edition of the Istat “Sample Survey on births” has been going on: it is a computer assisted telephone survey, where 15.000 mothers are interviewed. The advantages of C.A.T.I. survey versus different data collection techniques are well known, as well as their limits. In this paper we focus on specific strategies that can be adopted for minimizing both the sampling and the non-sampling errors when telephone survey method is adopted. As a matter of fact, C.A.T.I. surveys represent a cost and time saving way to collect socio-demographic data. At the same it is becoming increasingly difficult to ignore that less and less households have a fixed phone number and this can lead to an under-representation of selected population subgroups. By exploiting a previous Istat face-to-face survey and applying decision trees procedures we compared the characteristics of household with/without phone. Those results deserve high consideration in sample weights’ calculation to produce not-biased estimates across all groups, under-represented populations included. An other crucial aspect that must be considered is the “interviewer effect” that impacts both the response rate and data quality. Several tools are used to monitor and correct the interviewers’ behaviour. Particularly, a set of standardized indicators are daily computed both at aggregate and interviewer level. Moreover a continuous supervision of interviewers’ performance over a set of key-questions is assessed by multilevel regression analysis. If we recognize that interviewee are nested within interviewers, an intercept-only model can be estimated and the intra-class correlation coefficient represents the interviewer effect. Interviewers that show a random effect that significantly differs from zero are detected and further trained in order to raise data quality.