This article examines a critical issue in digital literacy assessment design when technological changes are happening with escalating speed in our society. There have been many assessment studies of digital literacy (DL) for diverse purposes and across different geographic and socioeconomic (geo-socioeconomic) contexts. While the assessment…

This paper scrutinizes the increasing trend of using international large-scale assessment (ILSA) data for causal inferences in educational research, arguing that such inferences are often tenuous. We explore the complexities of causality within ILSAs, highlighting the methodological constraints that challenge the validity of causal claims derived…

While methodologies are often presented as standardised procedures which, under specified conditions, should lead to the same conclusions, this case study presents the complex and deeply personal process of research (Addey and Piattoeva 2022). We analyse our application of network ethnography -- an approach presented by Ball (2016) -- to the study…

More than 20 years after the first round of the OECD's Programme for International Student Assessment (PISA), it has become one of the most important large-scale international assessments, at a global level and in particular in Europe. Thus, a growing number of scholars have examined or discussed its research design. Nonetheless, a key feature of…

Worldwide, the Programme for International Student Assessment (PISA) has emerged as a valid and reliable benchmark for assessing students' performance and for gaining significant influence over educational policy and decision-making. Since 2001, Qatar has engaged in massive educational reform, with high expectations for improving students'…

Background: In the context of large-scale educational assessments, the effort required to code open-ended text responses is considerably more expensive and time-consuming than the evaluation of multiple-choice responses because it requires trained personnel and long manual coding sessions. Aim: Our semi-supervised coding method eco (exploring…

The advancement of technology has led to a growing interest in assessing scientific inquiry within digital platforms. This shift towards dynamic and interactive inquiry assessments enables researchers to investigate not only the accuracy of student responses ("product data") but also their steps and actions leading to those responses…

In 2014, the OECD-PISA's Governing Board approved the addition of a set of global competence measures to its Programme of Student Assessment. In our paper, we explore whether and how there are discursive shifts between the two framing papers (2016/2018) and what the outcomes are for policy-shaping. To this end, we employ Network Text Analysis to…

Empirical research on teachers' professional knowledge is mostly conducted worldwide with reference to the construct 'pedagogical content knowledge' (PCK) introduced by Lee S. Shulman. In the 'Consensus Model' (CM) and the 'Refined Consensus Model' (RCM), PCK was further developed and differentiated. Nevertheless, neither PCK nor CM and RCM…

Errors that occasionally manifest in examination papers and other educational assessment instruments can threaten reliability and validity. For example, a multiple choice question could have two correct response options, or a geography question containing an inaccurate map could be unanswerable. In this paper we explore this oft-neglected element…

In the 21st century, international interaction in social, economic, cultural, and educational fields has increased. Consequently, international standards have become essential in national education policies, reforms, and practices. As an international assessment, PISA has started to function as a prominent tool in this regard. However, the impact…

Why are contractors keen to develop, implement, and analyse International Large-Scale Assessments (ILSAs) when they appear to make no financial gains? What, if not monetary profit, makes ILSA contracts so attractive? Almost all ILSA contractors interviewed as part of this project state that their ILSA work is an investment, either in the form of a…

International large-scale assessments such as PISA or PIAAC have started to provide public or scientific use files for log data; that is, events, event-related attributes and timestamps of test-takers' interactions with the assessment system. Log data and the process indicators derived from it can be used for many purposes. However, the intended…

Cosmopolitanism and nationalism are often presented in binary terms, as standing in opposition to each other, especially with regards to education. In this paper, we establish a framework for studying education policy today through the lens of a concept we have developed: 'cosmopolitan nationalism'. Many education systems around the world are…

This paper identifies and analyses the legitimation strategies used by the OECD as it expanded its role in global educational governance. Whilst the literature recognises the mainly discursive sources of legitimacy which the OECD derives from its testing regime, especially PISA, what remains unexplored is how exactly it has created the legitimacy…