This article examines a critical issue in digital literacy assessment design when technological changes are happening with escalating speed in our society. There have been many assessment studies of digital literacy (DL) for diverse purposes and across different geographic and socioeconomic (geo-socioeconomic) contexts. While the assessment…

This paper scrutinizes the increasing trend of using international large-scale assessment (ILSA) data for causal inferences in educational research, arguing that such inferences are often tenuous. We explore the complexities of causality within ILSAs, highlighting the methodological constraints that challenge the validity of causal claims derived…

While methodologies are often presented as standardised procedures which, under specified conditions, should lead to the same conclusions, this case study presents the complex and deeply personal process of research (Addey and Piattoeva 2022). We analyse our application of network ethnography -- an approach presented by Ball (2016) -- to the study…

More than 20 years after the first round of the OECD's Programme for International Student Assessment (PISA), it has become one of the most important large-scale international assessments, at a global level and in particular in Europe. Thus, a growing number of scholars have examined or discussed its research design. Nonetheless, a key feature of…

Worldwide, the Programme for International Student Assessment (PISA) has emerged as a valid and reliable benchmark for assessing students' performance and for gaining significant influence over educational policy and decision-making. Since 2001, Qatar has engaged in massive educational reform, with high expectations for improving students'…

This paper considers the inclusion of creative thinking as an innovative domain within the OECD's PISA (Program for International Student Assessment) and offers a series of reflections on the opportunities and limitations that follow from this. Although this recognition of the importance of creative thinking represents a step forward, the current…

This issue of Snapshots explores how student effort may vary depending on the perceived stakes of the assessment. We revisit the self-reported levels of effort that students invested in the PISA test and compare them with the effort they indicated they would have made if the results counted towards their school marks. Australia's results are…

Background: In the context of large-scale educational assessments, the effort required to code open-ended text responses is considerably more expensive and time-consuming than the evaluation of multiple-choice responses because it requires trained personnel and long manual coding sessions. Aim: Our semi-supervised coding method eco (exploring…

The advancement of technology has led to a growing interest in assessing scientific inquiry within digital platforms. This shift towards dynamic and interactive inquiry assessments enables researchers to investigate not only the accuracy of student responses ("product data") but also their steps and actions leading to those responses…

In 2014, the OECD-PISA's Governing Board approved the addition of a set of global competence measures to its Programme of Student Assessment. In our paper, we explore whether and how there are discursive shifts between the two framing papers (2016/2018) and what the outcomes are for policy-shaping. To this end, we employ Network Text Analysis to…

Empirical research on teachers' professional knowledge is mostly conducted worldwide with reference to the construct 'pedagogical content knowledge' (PCK) introduced by Lee S. Shulman. In the 'Consensus Model' (CM) and the 'Refined Consensus Model' (RCM), PCK was further developed and differentiated. Nevertheless, neither PCK nor CM and RCM…

The most recent OECD Programme for International Student Assessment (PISA) survey enables educators, policymakers and the wider community to compare Australian students with each other, as well as with their counterparts across the world. PISA measures the extent to which 15-year-old students near the end of compulsory education have acquired the…

Errors that occasionally manifest in examination papers and other educational assessment instruments can threaten reliability and validity. For example, a multiple choice question could have two correct response options, or a geography question containing an inaccurate map could be unanswerable. In this paper we explore this oft-neglected element…

In the 21st century, international interaction in social, economic, cultural, and educational fields has increased. Consequently, international standards have become essential in national education policies, reforms, and practices. As an international assessment, PISA has started to function as a prominent tool in this regard. However, the impact…

Why are contractors keen to develop, implement, and analyse International Large-Scale Assessments (ILSAs) when they appear to make no financial gains? What, if not monetary profit, makes ILSA contracts so attractive? Almost all ILSA contractors interviewed as part of this project state that their ILSA work is an investment, either in the form of a…