The present paper questions the content validity of the eight creativity-related self-report scales available in PISA 2022's context questionnaire and provides a set of considerations for researchers interested in using these indexes. Specifically, we point out some threats to the content validity of these scales (e.g., "creative thinking…

Background: The existence of a multiplier, compositional or social context effect is debated extensively in the literature on school effectiveness and also relates to the wider issue of equity in educational outcomes. However, comparatively little attention has been given to whether or not the association between student achievement and school…

Response process data have the potential to provide a rich description of test-takers' thinking processes. However, retrieving insights from these data presents a challenge for educational assessments and educational data mining as they are complex and not well annotated. The present study addresses this challenge by developing a computational…

Large-scale standardized tests are regularly used to measure student achievement overall and for student subgroups. These uses assume tests provide comparable measures of outcomes across student subgroups, but prior research suggests score comparisons across gender groups may be complicated by the type of test items used. This paper presents…

Item Response Theory (IRT) is a ubiquitous model for understanding humans based on their responses to questions, used in fields as diverse as education, medicine and psychology. Large modern datasets offer opportunities to capture more nuances in human behavior, potentially improving test scoring and better informing public policy. Yet larger…

In the social sciences, estimating causal effects is particularly difficult. Gold standards are set by randomized experiments in many cases expensive, unenforceable for ethical and practical reasons. Recent research has drawn attention to techniques that under some conditions, could estimate causal effects on non-experimental observable data. One…

This paper briefly discusses four measurement challenges of data science or "big data" in educational assessments that are enabled by technology: 1. Dealing with change over time via time-based data. 2. How a digital performance space's relationships interact with learner actions, communications and products. 3. How layers of…

Those who work in second- and foreign-language testing often find Koretz's concern for validity inferences under high-stakes (VIHS) conditions both welcome and familiar. While the focus of the article is more narrowly on the potential for two instructional responses to test-based accountability, "reallocation" and "coaching,"…

Recent results of international assessment programs (e.g., PISA) have shown a large difference in high school students' performance between northern and southern Italy. On this basis, it has been argued that the discrepancy reflects differences in average intelligence of the inhabitants of regions and is associated with genetic factors ( and ).…

Inferences from recent international comparative data on mathematical attainment, frequently quoted as justification for curriculum change, are critically examined, and the implications are contrasted with expressed curricular aims. Using characterisations by Ball (1990) and Ernest (1991), positions of key actors are analysed in relation to…

In this study it is aimed to describe and evaluate comparatively the reading literacy exam results, the finance of education and schools, and socio-cultural status of parents in Turkey and the top-five OECD countries, Finland, Korea, Canada, Australia, New Zealand respectively, in the light reports and publications by OECD regarding PISA 2003 and…

Background/Context: Two key uses of international assessments of achievement have been (a) comparing country performances for identifying the countries with the best education systems and (2) generating insights about effective policy and practice strategies that are associated with higher learning outcomes. Do country rankings really reflect the…