Probes are follow-ups to survey questions used to gain insights on respondents' understanding of and responses to these questions. They are usually administered as open-ended questions, primarily in the context of questionnaire pretesting. Due to the decreased cost of data collection for open-ended questions in web surveys, researchers have argued…

Recent years have seen a growing number of studies investigating the accuracy of nonprobability online panels; however, response quality in nonprobability online panels has not yet received much attention. To fill this gap, we investigate response quality in a comprehensive study of seven nonprobability online panels and three probability-based…

"Methods and Procedures in PIRLS 2016" documents the development of the Progress in International Reading Literacy Study (PIRLS) assessments and questionnaires and describes the methods used in sampling, translation verification, data collection, database construction, and the construction of the achievement and context questionnaire…

Although International Association for the Evaluation of Educational Achievement-pioneered international large-scale assessment (ILSA) of education is now a well-established science, non-practitioners and many users often substantially misunderstand how large-scale assessments are conducted, what questions and challenges they are designed to…

The "PISA 2012 Technical Report" describes the methodology underlying the PISA 2012 survey, which tested 15-year-olds' competencies in mathematics, reading and science and, in some countries, problem solving and financial literacy. It examines the design and implementation of the project at a level of detail that allows researchers to…

Two test items that examined high school students' beliefs of sample size for large populations using the context of opinion polls conducted prior to national and state elections were developed. A trial of the two items with 21 male and 33 female Year 9 students examined their naive understanding of sample size: over half of students chose a…

The Organisation for Economic Cooperation and Development's (OECD's) Programme for International Student Assessment (PISA) surveys, which take place every three years, have been designed to collect information about 15-year-old students in participating countries. PISA examines how well students are prepared to meet the challenges of the future,…

Three methods for the estimation of the reliability of single dichotomous items are discussed. All methods are based on the assumptions of nondecreasing and nonintersecting item response functions and the Mokken model of double monotonicity. Based on analytical and Monte Carlo studies, it is concluded that one method is superior to the other two…

The purpose of this investigation was to examine the item characteristics of the General Educational Development Tests (GED) for specific subgroups of students. The method employed was the approach suggested by Linn and Harnisch in which the three-parameter logistic model is used for detecting interactions between item content and group…

The validity of logistic test models and computer simulation methods for generating sampling distributions of item bias statistics was evaluated under the hypothesis of no item bias. Test data from 937 ninth-grade students were used to develop 7 steps for applying computer-simulated baseline statistics in test development. (SLD)

This report supplies details of the design and data analysis of the 1986 National Assessment of Educational Progress (NAEP) to allow the reader to judge the utility of the design, data quality, reasonableness of assumptions, appropriateness of data analyses, and generalizability of inferences made from the data. After an introduction by A. E.…

Research efforts have established that the Mantel-Haenszel procedure (MHP) is an effective method for detecting the presence of test items exhibiting differential item functioning (DIF). While the MHP has been advocated for situations where item response theory based methods may not be usable, recent findings have suggested that the performance of…

The 1992 National Assessment of Educational Progress (NAEP) monitored the performance of students in American schools in reading, mathematics, science, and writing. The sample consisted of more than 145,000 public and private school students in grades 4, 8, 11, and 12. This technical report provides details of instrument development, sample…

People who care about young people are aware of the serious problems caused by underage alcohol use. They should also be aware that there are many effective strategies for reducing underage drinking, and every state and community should be using these strategies. The "Guide to Conducting Youth Surveys" provides information about one important tool…