The purpose of this National Assessment of Educational Progress (NAEP) Achievement Levels Validity Argument Report is to synthesize evidence currently available to address the validity of the interpretations and uses of the NAEP Achievement Levels. Validity is the extent to which theory and evidence supports or refutes proposed and enacted test…

In many countries, standardized math tests are important for achieving academic success. Here, we examine whether content of items, the story that explains a mathematical question, biases performance of low-SES students. In a large-scale cohort study of Trends in International Mathematics and Science Studies (TIMSS)--including data from 58…

This paper introduces an approach that uses latent class analysis to identify cut scores (LCA-CS) and categorize respondents based on context scales derived from largescale assessments like PIRLS, TIMSS, and NAEP. Context scales use Likert scale items to measure latent constructs of interest and classify respondents into meaningful ordered…

Developing tools to identify students' misconceptions about basic biology concepts is necessary. Therefore, a two-tier diagnostic test was developed to determine such misconceptions in primary school (3rd-4th Grade) students. The test content includes two-tiered multiple-choice questions addressing common misconceptions found in the literature on…

Educators are often trying to foster Science, Technology, Engineering, and Math (STEM) engagement by facilitating interest in STEM content, but research suggests that STEM interest generally declines through the primary and secondary grades. To counter this trend, STEM interventions have begun to focus on younger students. To support this work, we…

Large scale international assessments depend on invariance of measurement across countries. An important consideration when observing cross-national differential item functioning (DIF) is whether the DIF actually reflects a source of bias, or might instead be a methodological artifact reflecting item response theory (IRT) model misspecification.…

Inference is considered an important factor in comprehension models and has been described as a causal factor in predicting comprehension. To date, specific tests for inference are rare and often rely on specific thematic texts. This reliance on thematic inference may raise some concerns as inference is related to prior text-specific knowledge.…

Since classroom climate is inherently a group level phenomenon, we argue that measures shown to be valid and reliable in multi-level settings are needed. We propose a brief, 5-item measure of classroom climate and demonstrate its reliability and validity in multi-level structural equation models. The measure demonstrated good reliability and we…

Computer-based testing is becoming dominant for assessments in education. In New York, students take state assessments, which are now administered digitally. While this transition in technology offers advantages, there are also challenges, including insufficient digital literacy for students to adequately meet the technological demands of the…

This paper extends existing work on teacher weighting in student-centered surveys by looking into aspects of practical implementation of deriving and using weights for teacher-centered analysis in the Trends in International Mathematics and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS). The formal…

The purpose of this study was to compare the classification accuracy of data produced by the previous year's end-of-year New York state assessment, a computer-adaptive diagnostic assessment ("i-Ready"), and the gating combination of both assessments to predict the rate of students passing the following year's end-of-year state assessment…

Focusing on the variables that can affect both academic achievement and the well-being of students has been crucial for their development, making schools effective and designing educational policy as well as curriculum. The study has aimed to investigate the effect of peer bullying on academic achievement and to determine moderators in the…

This study aims to apply the differential item functioning (DIF) technique with the deterministic inputs, noisy "and" gate (DINA) model to validate the mathematics construct and diagnostic attribute profiles across American and Singaporean students. Even with the same ability level, every single item is expected to show uniform DIF…

In this article, we extend Liao's test for across-group comparisons of the fixed effects from the generalized linear model to the fixed and random effects of the generalized linear mixed model (GLMM). Using as our basis the Wald statistic, we developed an asymptotic test statistic for across-group comparisons of these effects. The test can be…

International Large-Scale Assessments (ILSAs) have had significant impact on education policy across the globe. But has interest in ILSAs now started to wane? This paper presents new evidence on this issue, exploring how the amount of attention three major ILSAs receive compares across countries, between studies and over time. Using information on…