Questionnaires are by far the most common tool for measuring noncognitive constructs in psychology and educational sciences. Response bias may pose an additional source of variation between respondents that threatens validity of conclusions drawn from questionnaire data. We present a mixture modeling approach that leverages response time data from…

This study aims to assess student motivation to provide valid responses to the PISA student questionnaire. This was done by modeling response times using a three-component finite mixture model, comprising two satisficing response styles (rapid and idle) and one optimizing response style. Each participant's motivation was operationalized as their…

Purpose: Evidence from past surveys suggests that some interviewees simplify their responses even in very well-organized and highly respected surveys. This paper aims to demonstrate that some interviewers, too, simplify their task by at least partly fabricating their data, and that, in some survey research institutes, employees simplify their task…

Large-scale assessments (LSAs) use Mislevy's "plausible value" (PV) approach to relate student proficiency to noncognitive variables administered in a background questionnaire. This method requires background variables to be completely observed, a requirement that is seldom fulfilled. In this article, we evaluate and compare the…

The Programme for International Student Assessment (PISA) assesses the knowledge and skills in mathematics, reading and science of 15-year-old pupils in countries around the world. PISA is run by the Organisation for Economic Co-operation and Development (OECD), and assessment is typically undertaken every 3 years, allowing us to chart how…

School climate is one of the significant factors determining educational achievement. However, the lack of instruments to measure it has complicated the study of this concept in Russia. We review the history of the study of the concept of "school climate," and we discuss approaches to how it can be defined. We describe the most widely…

Background: This paper extends a recent study by Kaplan and Su ("J Educ Behav Stat" 41: 51-80, 2016) examining the problem of matrix sampling of context questionnaire scales with respect to the generation of plausible values of cognitive outcomes in large-scale assessments. Methods: Following Weirich et al. ("Nested multiple…

Cognitive interviewing (CI) provides a method of systematically collecting validity evidence of response processes for questionnaire items. CI involves a range of techniques for prompting individuals to verbalise their responses to items. One such technique is concurrent verbalisation, as developed in Think Aloud Protocol (TAP). This article…

International large-scale assessments such as the Programme for International Student Assessment (PISA) yield comparative indicators of student achievement in various competence domains. This article focuses on global competence as a suggested cross-curricular domain for the PISA 2018 study. The measurement of global competence is related to a…

Science excellence is associated not only with a student's inherent aptitude but also a range of contextual factors. The objective of this paper was to identify the most important contextual characteristics of top performers in scientific literacy, by simultaneously considering factors at the PISA questionnaire-based student, family, and school…

This study investigates consistent and inconsistent respondents to mixed-worded questionnaire scales in large-scale assessments. Mixed-worded scales contain both positively and negatively worded items and are universally applied in different survey and content areas. Due to the changing wording, these scales require a more careful reading and…

This paper explains the rationale for updating the OECD Programme for International Student Assessment (PISA) 2021 questionnaire on Information and Communication Technology (ICT) and shows how it covers policy topics of current relevance. After presenting key findings based on previous ICT-related PISA data, the paper provides a summary of the…

This article presents findings on the consequences of matrix sampling of context questionnaires for the generation of plausible values in large-scale assessments. Three studies are conducted. Study 1 uses data from PISA 2012 to examine several different forms of missing data imputation within the chained equations framework: predictive mean…

This study explores the level of congruency between students' ratings of their own emotional and behavioral strengths and difficulties, and their teachers' ratings of these factors, when compared amongst two neighboring countries, Estonia and Finland. Secondly, it investigates the level of agreement between sixth grade students' ratings and their…

The purpose of this study was to identify the multilevel latent classes for reading, mathematics, and science success of the students, who participated in the Programme for International Student Assessment (PISA) 2012 from Turkey and to determine the predictive ability of: i) students' perseverance; ii) their openness to problem solving; iii)…