This study examined which aspects of critical literacy are focused on in the reading literacy assessment for the Programme for International Student Assessment (PISA) 2018 and what kinds of texts are related to the critical literacy items in the test. Based on theory-oriented qualitative content analysis, critical literacy items in PISA…

Teachers lack effective curriculum-based instruments to assess their students' scientific competence that would provide information for modifying their inquiry instruction. The main purpose of this study was to develop and validate a Curriculum-Based Scientific Competence (CBSC) test to assess students' scientific competence in a 1-semester Grade…

This qualitative descriptive analysis aims to interpret the complexity of the science items that are integrated with environmental sustainability content. This research was conducted on science test items in the 2013 Curriculum student science textbooks released by the Ministry of Education and Culture of the Republic of Indonesia. This study…

The essential purpose of developing PISA-like questions is to train students' reasoning abilities so that students' ability to solve PISA-like questions increases. Therefore, this research aims to produce PISA-like math problems in the context of historical buildings in the Karawang Regency that are valid, practical, and potentially affect…

This study aims to develop a math problem type PISA to familiarize students using problems with PISA standard and produces a valid, practical, PISA context of Bangka (Tanjung Kalian Lighthouse) type and see the Basic Math Skills (BMS) seen from the context of Tanjung Kalian lighthouse. The research method used is design research with the type of…

Purpose: Certain changes occurred in measurement and evaluation systems, centralized exams and exams implemented in schools in alignment with developments in the world. Our mean scores in exams including PISA and TIMMS were lower than other countries. This led the Ministry of Education to introduce skills-based questions in exams with the purpose…

Standardized assessments are widely used to determine access to educational resources with important consequences for later economic outcomes in life. However, many design features of the tests themselves may lead to psychological reactions influencing performance. In particular, the level of difficulty of the earlier questions in a test may…

This paper describes and evaluates a multistage adaptive testing (MSAT) design that was implemented for the Programme for International Student Assessment (PISA) 2018 main survey for the major domain of Reading. Through a simulation study, recovery of item response theory model parameters and measurement precision were examined. The PISA 2018 MSAT…

Efficiency in reading component skills is crucial for reading comprehension, as efficient subprocesses do not extensively consume limited cognitive resources, making them available for comprehension processes. Cognitive efficiency is typically measured with speeded tests of relatively easy items. Observed responses and response times indicate the…

Background: This paper extends a recent study by Kaplan and Su ("J Educ Behav Stat" 41: 51-80, 2016) examining the problem of matrix sampling of context questionnaire scales with respect to the generation of plausible values of cognitive outcomes in large-scale assessments. Methods: Following Weirich et al. ("Nested multiple…

In line with the widespread proliferation of digital technology in everyday life, many countries are now beginning to use computer-based exams (CBEs) in their post-primary education systems. To ensure that these CBEs are delivered in a manner that preserves their fairness, validity, utility and credibility, several factors pertaining to their…

Since the 1960s, the advocacy of "teaching mathematics so as to be useful" is not without hindrance in school curricula, partly due to the lack of appropriate assessment tools. Practical approaches have been accumulating quickly, but researchers showed that they are not satisfactory in testing students' ability to apply mathematical…

The fact that Turkey's achievement in large-scale international examinations is not at the expected level is associated with such factors as students, teachers, schools, curricula and assessment approaches. The literature links student-related causes mainly to their lack of higher-order thinking skills; and teacher-related causes to incompetency…

We investigated whether consensus-based test translation review procedures can be used effectively in cultural contexts with high social stratification. We staged two test translation review panels in China -- where social stratification may potentially inhibit individuals' ability to express opinion and disagreement. We adapted a consensus-based…

The OECD is planning to enhance existing PISA assessment instruments in reading, mathematics and science so that they will be more suitable to the context of developing countries. The main purpose of this paper is to identify the main technical issues associated with this aim. The paper reports detailed analysis of the existing PISA item pool and…