The purpose of this study was to apply two types of Differential Item Functioning (DIF), net and global DIF, as well as the framework of Differential Step Functioning (DSF) to real testing data to investigate measurement invariance related to test language. Data from the Program for International Student Assessment (PISA)-2006 polytomously scored…

This article advances the enlightened discussion of the nature, logic, and possible effects of the Programme for International Student Assessment (PISA). The purpose is to analyse the assumptions regarding how PISA is to achieve its intended effects, that is, to reconstruct PISA's programme theory (PT) and to probe the validity of its underlying…

This article explores the relationship between commensuration and affect in various contexts of education policy. Commensuration is the process through which disparate qualities are transformed into a common metric and is central to the production of performance data. The rise of governance through numbers in education has resulted in a…

Methods for detecting differential item functioning (DIF) and item bias are typically used in the process of item analysis when developing new measures; adapting existing measures for different populations, languages, or cultures; or more generally validating test score inferences. In 2007 in "Language Assessment Quarterly," Zumbo…

Using an empirically-based simulation study, we show that typically used methods of choosing an item calibration sample have significant impacts on achievement bias and system rankings. We examine whether recent PISA accommodations, especially for lower performing participants, can mitigate some of this bias. Our findings indicate that standard…

Background/Context: Because metacognitive knowledge includes knowledge about adequate learning strategies, and because an effective use of learning strategies is associated with higher levels of performance, substantial relationships can be assumed between metacognitive knowledge, strategic behavior, and performance. However, such a pattern of…

This article demonstrates how meta-analytic techniques, that have typically been used to synthesize findings across numerous studies, can also be applied to examine the reasons why relationships between background characteristics and outcomes may vary across different locations in a single multi-site survey. This application is particularly…

The OECD's Programme for International Student Assessment (PISA) enables participating countries to monitor 15-year old students' progress in reading, mathematics, and science literacy. The present study investigates persistent factors that contribute to science performance of Turkish students in PISA 2006 and PISA 2009. Additionally, the study…

The article reviews research and findings on problems and issues faced when translating international academic achievement tests. The purpose is to draw attention to the problems, to help to develop the procedures followed when translating the tests, and to provide suggestions for further research. The problems concentrate on the following: the…

The release of the 2012 results from the Programme for International Student Assessment (PISA) on December 3, 2012, by the Organisation for Economic Co-operation and Development ([OECD], 2013) brought about another round of public education fear mongering on the part of bureaucrats and policymakers. However, weaknesses with the PISA test design,…

Gauging the effectiveness of specific teaching strategies remains a major topic of interest in science education. Inquiry teaching among others has been supported by extensive research and recommended by the National Science Education Standards. However, most of the empirical evidence in support was collected in research settings rather than in…

The main purpose of this study is to improve estimation efficiency in obtaining maximum marginal likelihood estimates of contextual effects in the framework of nonlinear multilevel latent variable model by adopting the Metropolis-Hastings Robbins-Monro algorithm (MH-RM). Results indicate that the MH-RM algorithm can produce estimates and standard…

The main purpose of this study is to improve estimation efficiency in obtaining maximum marginal likelihood estimates of contextual effects in the framework of nonlinear multilevel latent variable model by adopting the Metropolis-Hastings Robbins-Monro algorithm (MH-RM; Cai, 2008, 2010a, 2010b). Results indicate that the MH-RM algorithm can…

Over the past three decades, the number of developing countries participating in International Large-Scale Assessments (ILSAs) has increased dramatically, while developing countries' usage of data from ILSAs for informing educational policy has not been fully realised. In this paper the authors argue that for ILSAs to be useful policy tools,…

Through a sociological analysis of the knowledge and actors that have become central to international assessments, the paper focuses on the processes that influence the production of shared narratives and agendas, adopting the position that their existence is not organic, but rather the product of undertakings that often fabricate and manage,…