Purpose: Given the scientific and practical difficulties inherent in measuring and comparing socioeconomic deprivation (SED), and the further complexity added in cross national measurements, the main aim of this paper was to check the validity of SED measures used in PISA 2018 dataset. The SED measure used in PISA 2018 was the PISA index of…

International large-scale assessments have gained much attention since the beginning of the twenty-first century, influencing education legislation in many countries. This includes Spain, where they have been used by successive governments to justify education policy change. Unfortunately, there was a problem with the PISA 2018 reading scores for…

International large-scale assessments provide cross-national data on students' cognitive and non-cognitive characteristics. A critical methodological issue that often arises in comparing data from cross-national studies is ensuring measurement invariance, indicating that the construct under investigation is the same across the compared groups.…

Socioeconomic status (SES) is a key covariate in analyses by the Programme for International Student Assessment (PISA). In its technical documentation, PISA offers evidence for the Economic, Social, and Cultural Status instrument (ESCS) as a valid measure of SES. This dissertation, however, offers both conceptual and empirical arguments that ESCS…

Mathematics modelling is a vital competency for students of all ages. In this study, we aim to fill the research gap about valid and reliable tools for assessing and grading mathematical modeling problems, particularly those reflecting multiple steps of the modelling cycle. We present in this paper the design of a reliable and valid assessment…

Research on methods for measuring examinee engagement with constructed-response items is limited. The present study used data from the PISA 2018 Reading domain to construct and compare indicators of test-taking effort on constructed-response items: response time, number of actions, the union (combining effortless responses detected by either…

Achievement tests are commonly used in education to evaluate students' academic performance and proficiency in specific subject areas. However, there is a major problem that threatens the validity of achievement test scores which is test-taking disengagement. Respondents provide answers that are inconsistent with their true ability level and can…

The waning interest of students in science became a global concern. The purpose of this research was to translate, adapt, and validate the My Attitude toward Science [MATS] questionnaire instrument, which was used to measure students' attitudes toward science in the Indonesian context. We also investigated the items that contributed to gender and…

This study aims to assess student motivation to provide valid responses to the PISA student questionnaire. This was done by modeling response times using a three-component finite mixture model, comprising two satisficing response styles (rapid and idle) and one optimizing response style. Each participant's motivation was operationalized as their…

The aim of this study is to develop a rubric to assess mathematical reasoning competence. Since the aim is to assess a competency, the frameworks of the PISA exams in the literature, which give an important place to competencies, have been examined. Due to its focus and in-depth analysis of mathematical reasoning, each of the actions expected from…

The Standards for Educational and Psychological Testing identify several strands of validity evidence that may be needed as support for particular interpretations and uses of assessments. Yet assessment validation often does not seem guided by these Standards, with validations lacking a particular strand even when it appears relevant to an…

The importance of the opportunity to learn (OTL) for mathematics achievement has been extensively researched. However, there were still unanswered questions regarding OTL's measurement, analytical level, and relationship with motivational beliefs. To fill in the gaps, we aimed to (1) scrutinize the reliability and validity of OTL, (2) investigate…

Drawing on Kane's argument-based approach to validity and Toulmin's later work on cosmopolitanism and diversity, this paper asks whose validity arguments and evidence are being presented in International Large-Scale Assessments (ILSAs), where and when. With a case study of the OECD's PISA for Development, we demonstrate that validity arguments are…

As a relevant cognitive-motivational aspect of ICT literacy, a new construct "ICT Engagement" is theoretically based on self-determination theory and involves the factors ICT interest, Perceived ICT competence, Perceived autonomy related to ICT use, and ICT as a topic in social interaction. In this manuscript, we present different…

PISA (Program for International Student Assessment) tests have enabled the OECD countries to see not only the success of their students in gaining the ability to solve some daily problems they may encounter in their lives but also the place in the world rankings as a result of an objective evaluation comparing the achievement results of…