The Impact of Time and Timing on the Survey Process and Data Quality.

Surveys are a tool to describe a pre-determined population in terms of socio-demographic characteristics, behaviors, attitudes, or opinions. The process of surveying a population can be decomposed into many sub-processes: 1. the preparation processes - developing a questionnaire and drawing a random sample of the population, 2. the fieldwork processes - making contacts with sample members and gaining their cooperation and interviewing the respondents, 3. data processing. Each sub-process can lead to errors which we classified in : a. nonobservation errors: 1. coverage and sampling errors, 2. nonresponse errors and 3. adjustment errors and, b. measurement errors: 1. specification errors, 2. measurement errors during the interview/ questionnaire completion and 3. data processing errors. The data that describe these processes are called paradata. Example of paradata are sequences of visits to a household to gain cooperation or the interview duration. The aim of this thesis is to study the relation between time-related paradata and the quality of the survey process, ultimately linked to the quality of the survey data. We restrict ourselves to face-to-face surveys, and our empirical analyses are all based on the data from the European Social Survey. We also focus on the fieldwork component of the survey process. In the first part of the thesis, we study the contacting and gaining cooperation part of the fieldwork. Therefore, we define the fieldwork power as the fieldwork production per time unit: the (standardized) number of contacts/completed interviews per week or the ratio of completed interviews to the number of contact attempts/ refusals per week. Each week of the fieldwork is associated with measures of the fieldwork power. The evolution or the dynamic of the fieldwork power is then modeled with a multi-level model, the country-round combinations of the European Social Survey as level 2 and the fieldwork power measurements as level 1. The results show that the general shape is cubic with a peak in the first 6 to 8 weeks, followed by a tail of lower fieldwork power. The shape is dependent on some survey design characteristics, such as the survey length, the type of sampling frame (individual vs non-individual), the percentage of refusal conversions or the number of active interviewers. As a follow-up on these results we simulated how the results from the fieldwork power model could be used to monitor a subsequent round of the European Social Survey, and linked the weekly fieldwork power to effort and data quality metrics. In the second part, we concentrated on the interview process itself using the interview speed, the number of applicable items per minute, as a proxy for the interviewer respondent interaction. Our findings show that, overall, in countries that participated in round 7 of the European Social Survey and in which the interview was computer assisted, interviewer effects on substantive variables are higher among fast and slow interviews compared to moderate interviews. Further decomposing the correlation between interview speed and the tendency to straight-line into respondent and interviewer levels, we found a positive correlation (significant at the 0.05 level) at the respondent-level in nine out of 15 countries and at the interviewer level in five countries. The fieldwork power and the interview speed are relevant and appropriate quality indicators to be used to monitor the fieldwork. If these two indicators deviates from expected values (which could be modeled from previous rounds, derived from experience or in a laboratory setting), the processes that they describe may not flow as anticipated. This could be a threat to the survey data quality, more specifically in terms of nonresponse errors and measurement errors.

Publication year:2019

Accessibility:Open