The present study explores the sources of validity evidence of the mathematics self-concept construct in Programme for International Student Assessment (PISA) 2022 with a focus on concurrent, discriminant, and predictive evidence. Mathematics self-concept was provided with a single index variable in PISA 2022. Using only the U.S. students who…

Creative activities are typically thought to be positively associated with creative ability, whether because more creative individuals select into creative activities, or because those activities support the development of creativity, or both. However, the PISA 2022 creative thinking report revealed an unexpected finding: Creative ability was…

Purpose: Given the scientific and practical difficulties inherent in measuring and comparing socioeconomic deprivation (SED), and the further complexity added in cross national measurements, the main aim of this paper was to check the validity of SED measures used in PISA 2018 dataset. The SED measure used in PISA 2018 was the PISA index of…

International large-scale assessments have gained much attention since the beginning of the twenty-first century, influencing education legislation in many countries. This includes Spain, where they have been used by successive governments to justify education policy change. Unfortunately, there was a problem with the PISA 2018 reading scores for…

International large-scale assessments provide cross-national data on students' cognitive and non-cognitive characteristics. A critical methodological issue that often arises in comparing data from cross-national studies is ensuring measurement invariance, indicating that the construct under investigation is the same across the compared groups.…

Socioeconomic status (SES) is a key covariate in analyses by the Programme for International Student Assessment (PISA). In its technical documentation, PISA offers evidence for the Economic, Social, and Cultural Status instrument (ESCS) as a valid measure of SES. This dissertation, however, offers both conceptual and empirical arguments that ESCS…

Mathematics modelling is a vital competency for students of all ages. In this study, we aim to fill the research gap about valid and reliable tools for assessing and grading mathematical modeling problems, particularly those reflecting multiple steps of the modelling cycle. We present in this paper the design of a reliable and valid assessment…

Research on methods for measuring examinee engagement with constructed-response items is limited. The present study used data from the PISA 2018 Reading domain to construct and compare indicators of test-taking effort on constructed-response items: response time, number of actions, the union (combining effortless responses detected by either…

Achievement tests are commonly used in education to evaluate students' academic performance and proficiency in specific subject areas. However, there is a major problem that threatens the validity of achievement test scores which is test-taking disengagement. Respondents provide answers that are inconsistent with their true ability level and can…

The waning interest of students in science became a global concern. The purpose of this research was to translate, adapt, and validate the My Attitude toward Science [MATS] questionnaire instrument, which was used to measure students' attitudes toward science in the Indonesian context. We also investigated the items that contributed to gender and…

The present paper questions the content validity of the eight creativity-related self-report scales available in PISA 2022's context questionnaire and provides a set of considerations for researchers interested in using these indexes. Specifically, we point out some threats to the content validity of these scales (e.g., "creative thinking…

This study aims to assess student motivation to provide valid responses to the PISA student questionnaire. This was done by modeling response times using a three-component finite mixture model, comprising two satisficing response styles (rapid and idle) and one optimizing response style. Each participant's motivation was operationalized as their…

The aim of this study is to develop a rubric to assess mathematical reasoning competence. Since the aim is to assess a competency, the frameworks of the PISA exams in the literature, which give an important place to competencies, have been examined. Due to its focus and in-depth analysis of mathematical reasoning, each of the actions expected from…

The Standards for Educational and Psychological Testing identify several strands of validity evidence that may be needed as support for particular interpretations and uses of assessments. Yet assessment validation often does not seem guided by these Standards, with validations lacking a particular strand even when it appears relevant to an…

The importance of the opportunity to learn (OTL) for mathematics achievement has been extensively researched. However, there were still unanswered questions regarding OTL's measurement, analytical level, and relationship with motivational beliefs. To fill in the gaps, we aimed to (1) scrutinize the reliability and validity of OTL, (2) investigate…