Large-scale assessments are rich sources of data that can inform a diverse range of research questions related to educational policy and practice. For this reason, datasets from large-scale assessments are available to enable secondary analysts to replicate and extend published reports of assessment results. These datasets include multiple imputed…

For large-scale assessments, data are often collected with missing responses. Despite the wide use of item response theory (IRT) in many testing programs, however, the existing literature offers little insight into the effectiveness of various approaches to handling missing responses in the context of scale linking. Scale linking is commonly used…

The process of quantification is a powerful development shaping many domains of life today. In the area of education, for example, performance measurement, testing and ranking have become common tools of governance. Quantification is not a neutral way of describing society, but a process of valorisation. It has three sociologically relevant…

Promoting understanding of the epistemologies of science has long been the primary objective in science education, and can be viewed as a form of science learning outcome. Many studies have attempted to understand learners' conceptions of epistemology in science from various perspectives and methods; however, no recent reviews have focused on the…

The arc of quantitative educational research should not be etched in stone but should adapt and change over time. In this article, I argue that it is time for a reshaping by offering my personal view of the past, present and future of our field. Educational research--and research in the social and life sciences--is at a crossroads. There are many…

Measuring race and ethnicity for administrative data sets and then analyzing these data to understand racial/ethnic disparities present many logistical and theoretical challenges. In this chapter, we conduct a synthetic review of studies on how to effectively measure race/ethnicity for administrative data purposes and then utilize these measures…

We review the GIDNA and CDM packages in R for fitting cognitive diagnosis/diagnostic classification models. We first provide a summary of their core capabilities and then use both simulated and real data to compare their functionalities in practice. We found that the most relevant routines in the two packages appear to be more similar than…

Organizational relationships are complex. Employees do their work as individuals, but also as members of larger teams. They exist within various social networks, both within and spanning organizations. Multilevel theory is at the core of the organizational sciences, and unpacking multilevel relationships is fundamental to the challenges faced…

This article presents findings on the consequences of matrix sampling of context questionnaires for the generation of plausible values in large-scale assessments. Three studies are conducted. Study 1 uses data from PISA 2012 to examine several different forms of missing data imputation within the chained equations framework: predictive mean…

Modeling the distribution of site level effects is an important problem, but it is also an incredibly difficult one. Current methods rely on distributional assumptions in multilevel models for estimation. There it is hoped that the partial pooling of site level estimates with overall estimates, designed to take into account individual variation as…

We offer an introduction to the five papers that make up this special section. These papers deal with a range of the methodological challenges that face researchers analyzing fMRI data--the spatial, multilevel, and longitudinal nature of the data, the sources of noise, and so on. The papers all provide analyses of data collected by a multi-site…

Background: In order to measure the proficiency of person populations in various domains, large-scale assessments often use marginal maximum likelihood IRT models where person proficiency is modelled as a random variable. Thus, the model does not provide proficiency estimates for any single person. A popular approach to derive these proficiency…

In an age of assessment and accountability, academic libraries feel much pressure to prove their value according to new university measurements of student success. This study describes a methodology for how libraries may examine student interactions with services to assess whether library usage impacts student grade point averages (GPAs) and…

This is a keynote presentation given at AERA on developing a validity agenda for growth models in a large scale (e.g., state) setting. The emphasis of this presentation was to indicate that growth models and the validity agenda designed to provide evidence in supporting the claims to be made need to be personalized to meet the local or…

It is common to assume during a statistical analysis of a multiscale assessment that the assessment is composed of several unidimensional subtests or that it has simple structure. Under this assumption, the unidimensional and multidimensional approaches can be used to estimate item parameters. These two approaches are equivalent in parameter…