Multidimensional achievement tests are recently gaining more importance in educational and psychological measurements. For example, multidimensional diagnostic tests can help students to determine which particular domain of knowledge they need to improve for better performance. To estimate the characteristics of candidate items (calibration) for…

Combining methods from earlier content validity and more contemporary content alignment studies may allow a more complete evaluation of the meaning of test scores than if either set of methods is used on its own. This article distinguishes item relevance indices in the content validity literature from test representativeness indices in the…

A common belief underlying the dimensional comparison theory is that reading self-concept has a positive relation to reading achievement but a negative relation to math achievement. However, this belief is incomplete and potentially misleading as supporting evidence is usually derived from studies without simultaneously considering the mediated…

Globalization and its impact on contemporary societies have gained new impetus with the notions of global citizenship education (GCED) and education for sustainable development (ESD), considered, together with civic and citizenship education (CCE), as a means for promoting students' engagement in global/local issues and providing them with the…

This study investigates the effect of survey order in measuring self-reported cognitive load. Understanding how survey order influences responses is crucial, but it has been largely overlooked in the context of cognitive load. Using a 2 × 2 experimental design with 319 high school students, the study manipulated intrinsic cognitive load (ICL)…

Universal screening in reading is a common, and often required, practice in early elementary school. Computer-adaptive screening tools, such as "Istation's Indicators of Progress-Early Reading" (ISIP-ER), are often chosen for this purpose in schools. In our present study, we examine the validity evidence between the ISIP-ER in…

Stratification is an important design feature of many studies using complex sampling designs and it is often used in large-scale assessment (LSA) studies, such as the "Programme for International Student Assessment" (PISA), for two main reasons. First, stratification variables that achieve a high between and low within strata variance…

The Programme for International Student Assessment (PISA) has instigated educational changes and reforms globally, in particular, introducing and intensifying neoliberal logic and governance in test-taking countries and beyond. PISA outcome impacted upon the educational governance of South Korea as well, however, the changes deviated from what…

This quantitative study aimed to research factors that influence third-graders' reading performance on the State of Texas Assessments of Academic Readiness, a standardized test used to measure the proficiency of students in various subjects. The independent variables of this study were student attendance, teacher's professional background, and…

Grade retention is at the core of the education debate in Spain, to the extent that its impact on students' competences has not been assessed beyond correlation. Because of that, in the present study, we analyse the influence of grade retention on students' competences, using more than 146,000 students from 6 PISA cycles (2003-2018) and an…

The article describes practical suggestions for measurement researchers and psychometricians to respond to calls for social responsibility in assessment. The underlying assumption is that personalizing large-scale assessment improves the chances that assessment and the use of test scores will contribute to equity in education. This article…

Traditional estimators of reliability such as coefficients alpha, theta, omega, and rho (maximal reliability) are prone to give radical underestimates of reliability for the tests common when testing educational achievement. These tests are often structured by widely deviating item difficulties. This is a typical pattern where the traditional…

Increased use of computer-based assessments has facilitated data collection processes that capture both response product data (i.e., correct and incorrect) and response process data (e.g., time-stamped action sequences). Evidence suggests a strong relationship between respondents' correct/incorrect responses and their problem-solving proficiency…

This study examines potential assessment bias based on students' primary language status in PISA 2018. Specifically, multilingual (MLs) and nonmultilingual (non-MLs) students in the United States are compared with regard to their response time as well as scored responses across three cognitive domains (reading, mathematics, and science).…

The OECD's PISA program assesses 15-year-old students globally in key competencies every 3 years, providing influential data on education quality and spurring policy debates. In the latest cycle, the innovation domain focused on creative thinking, assessing over 140,000 students across 60+ countries, in the largest study of adolescent creativity…