The Programme for International Student Assessment (PISA) introduced creative thinking as an innovative domain in 2022. This paper examines the unique methodological issues in international assessments and the implications of measuring creative thinking within PISA's framework, including stratified sampling, rotated form designs, and a distinct…

Randomized experiments are considered the gold standard for causal inference because they can provide unbiased estimates of treatment effects for the experimental participants. However, researchers and policymakers are often interested in using a specific experiment to inform decisions about other target populations. In education research,…

The Programme for International Assessment of Adult Competencies (PIAAC) will establish technical standards and guidelines to ensure that the survey design and implementation processes of PIAAC yield high-quality and internationally comparable data. This document provides a revised version of the technical standards and guidelines originally…

This study examined the percentage of task-sampling variability in performance assessment via a meta-analysis. In total, 50 studies containing 130 independent data sets were analyzed. Overall results indicate that the percentage of variance for (a) differential difficulty of task was roughly 12% and (b) examinee's differential performance of the…

Large scale surveys usually employ a complex sampling design and as a consequence, no standard methods for estimation of the standard errors associated with the estimates of population means are available. Resampling methods, such as jackknife or bootstrap, are often used, with reference to their properties of robustness and reduction of bias. A…

This report describes a statistically designed experiment that was carried out in an operational setting to determine the contributions of different sources of variation to the unreliability of scoring. The experiment made novel use of partially balanced incomplete block designs that facilitated the unbiased estimation of certain main effects…

The Third International Mathematics and Science Study (TIMSS) developed and administered tests and questionnaires in three student populations to document the quality of mathematics and science education in 45 participating countries. Study design, instrument development, and research procedures were achieved through a complex collaborative…

This report supplies details of the design and data analysis of the 1986 National Assessment of Educational Progress (NAEP) to allow the reader to judge the utility of the design, data quality, reasonableness of assumptions, appropriateness of data analyses, and generalizability of inferences made from the data. After an introduction by A. E.…

Details of the design and data analysis associated with the 1988 National Assessment of Educational Progress (NAEP) are presented. The 1988 NAEP surveyed American students' knowledge of reading, writing, civics, U.S. history, and geography. Small-scale studies were conducted for mathematics and science. Populations included public school students…

The 1992 National Assessment of Educational Progress (NAEP) monitored the performance of students in American schools in reading, mathematics, science, and writing. The sample consisted of more than 145,000 public and private school students in grades 4, 8, 11, and 12. This technical report provides details of instrument development, sample…