From a scientific point of view, it is essential that we establish knowledge about the measurement quality of survey data. For years, survey researchers have argued with their critics about the quality of survey data without any strong factual basis, but new data resources and modeling strategies now make it possible to assess the quality of survey measures.
This website provides public access to information from a database on the Reliability of Survey Measures (RSM). The RSM database was developed to investigate the quality of data obtained via the survey interview as a method of gathering social data. The website contains information focusing on an approach to evaluating one aspect of the quality of survey data—the reliability or consistency of measurement. Please click below for further information on the background and history of the project.
Reliability of measurement is the sine qua non for research investigations in all sciences. Without reliable measures one cannot achieve valid empirical results. It is a necessary condition for the validity of all measurement, and the quality concerns about survey measurement. The growing recognition that measurement errors pose a serious limitation to the validity and usefulness of the information collected in survey research signals the importance of the present research.
There has been much written about the sources of measurement errors in surveys and best practices in developing high quality survey questionnaires. The overarching goal of the present project is to establish more of a factual basis for conjectures that exist in the survey methods literature concerning the attributes of good survey questions, using reliability as an important criterion. The project draws upon the insights of the burgeoning research literature on survey methodology developed over the past two decades, which has devoted new attention to the information gathering process in surveys.
Through an analysis of the reliability information and the attributes of survey questions from several large-scale panel studies, practical conclusions may be drawn about the association between attributes of survey questions and the reliability of measurement. In the long run, such a knowledge-base will improve the quality of survey data and survey measurement practices.
This website provides information on the methods we employ and the sources of the data we use to estimate the reliability of survey measures. We also provide a list of publications (books, chapters of books, and journal articles), to which the reader may turn for more extensive information concerning the project and our approach to reliability estimation.
In addition, the website contains information from a data base of roughly 1,350 questions representative of typical questions used in social science surveys. The data base contains estimates of question-specific reliabilities, along with detailed coding of attributes of the questions, which can be used to evaluate the optimal properties of survey questions with respect to levels of measurement error.
Background/History