This study uses evidence from a longitudinal survey (PISA Plus, Germany) to examine the potential of bias in international large-scale assessments (ILSAs). In PISA Plus, participation was mandatory at the first measurement point, but voluntary at the second measurement point. The study provides evidence for relevant selection bias regarding…

Questionnaires are by far the most common tool for measuring noncognitive constructs in psychology and educational sciences. Response bias may pose an additional source of variation between respondents that threatens validity of conclusions drawn from questionnaire data. We present a mixture modeling approach that leverages response time data from…

Social institutions function not only by reproducing specific practices but also by reproducing discourses endowing such practices with meaning. The latter in turn is related to the development of the identities or subjectivities of those who live and thrive within such institutions. Meaning and subjectivity are therefore significant sociological…

The aim of this study is to investigate the cross-cultural measurement invariance of the reading skills items of the PISA 2018 at test and item level. Another aim of the study is to determine the item bias for items that do not show cross-cultural measurement invariance in line with expert opinions. A survey model was used in the study. The study…

In a signal detection theory (SDT) approach to multiple choice exams, examinees are viewed as choosing, for each item, the alternative that is perceived as being the most plausible, with perceived plausibility depending in part on whether or not an item is known. The SDT model is a process model and provides measures of item difficulty, item…

Facing increasing critique that PISA focuses too narrowly on cognitive achievement and human/knowledge capital, the OECD has recently shifted some of its focus to student happiness. The 2017 Students' Well-Being report distinguishes between 'happy schools' and 'unhappy schools', showing that among students who combined high performance and life…

The development of international surveys on children's learning like PISA, PIRLS and TIMSS--delivering comparable achievement measures across educational systems--has revealed large cross-country variability in average performance and in the degree of inequality across social groups. A key question is whether and how institutional differences…

We examine construct validity of the anchoring method used with 12 noncognitive scales from the Programme for International Student Assessment (PISA) 2012 project. This method combines individuals' responses to vignettes and self-rated scores based on Likert-type items. It has been reported that the use of anchoring vignettes can reverse…

This study investigates knowledge structures and scientific communication using bibliometric methods to explore scientific knowledge production and dissemination. The aim is to develop knowledge about this growing field by investigating studies using international large-scale assessment (ILSA) data, with a specific focus on those using Programme…

Drawing on a sociocultural perspective on educational assessment the empirical examples of Margareta Serder and Anders Jakobsson serve as a starting point for a critical analysis of PISA and the image of science education it perpetuates. While PISA claims to neutrally measure competencies relevant to science education, I argue that the test…

In the twenty-first century, the use of standardized tests as the primary means to evaluate schools and teachers in the United States has contributed to severe dilemmas, including misleading information on what students know, lower-level instruction, cheating, less collaboration, unfair treatment of teachers, and biased teaching. This article…

Pundits and bureaucrats use the results from international tests, particularly the PISA, to make claims about the quality of the public education system in the United States and make policy recommendations. In this article I argue, with evidence, that the scores and rankings from PISA are not important and that they cannot give policy makers or…

The aim of the present article is to point out the problematic issue of administrating PISA science literacy exam to the ultra-orthodox schools in Israel. It has been assumed that some texts included in the test may offend the feelings of the ultra-orthodox population or may contradict Orthodox upbringing and therefore constitute a cultural bias.

Large scale surveys in education have to face non-response issues that might bias the results. Non-response can occur at three levels: (i) a school refuses to participate, (ii) a sample student fails to participate and (iii) a participating student refuses to answer a particular question. Until now schools and student non-response have been…