Changepoints are abrupt variations in a sequence of data in statistical inference. In educational and psychological assessments, it is pivotal to properly differentiate examinees' aberrant behaviors from solution behavior to ensure test reliability and validity. In this paper, we propose a sequential Bayesian changepoint detection algorithm to…

Purpose: The study examined age-related differences in the use of semantic context and in the effect of semantic competition in spoken sentence processing. We used offline (response latency) and online (eye gaze) measures, using the "visual world" eye-tracking paradigm. Method: Thirty younger and 30 older adults heard sentences related…

Two experiments tracked the encoding of relational information (actions at the level of the prelinguistic message and verbs at the level of the sentence) during formulation of transitive event descriptions (e.g., The tiger is scratching the photographer). At what point during message and sentence formulation do speakers encode actions and verbs?…

The aim of this paper is to describe, analyze, and discuss how Swedish schools and the national tests in schools, university teaching and examination, and the college admissions test, Swedish Scholastic Aptitude Test (SweSAT), have been affected by the COVID-19 situation. A further aim is to discuss the challenges in schools, universities and in…

In 2015, the "Programme for International Student Assessment" (PISA) introduced multiple changes in its study design, the most extensive being the transition from paper- to computer-based assessment. We investigated the differences between German students' text responses to eight reading items from the paper-based study in 2012 to text…

In experiments with univalent target stimuli, task-switching costs can be eliminated if participants are unaware of the task rules and apply cue-target-response associations. However, in experiments with bivalent target stimuli, participants show task-switching costs. Participants may exhibit switch costs even when no task rules are provided in…

This paper reports on a pilot study that used eye tracking techniques to make detailed observations of item response processes in the OECD Programme for the International Assessment of Adult Competencies (PIAAC). The lab-based study also recorded physiological responses using measures of pupil diameter and electrodermal activity. The study tested…

Approximately a third of the Programme for International Student Assessment (PISA) items in the core domains (math, reading, and science) are constructed-response items and require human coding (scoring). This process is time-consuming, expensive, and prone to error as often (a) humans code inconsistently, and (b) coding reliability in…

TIMSS (Trends in International Mathematics and Science Study) is a long-standing international assessment of mathematics and science at the fourth and eighth grades that has been collecting trend data every four years since 1995. About 70 countries use TIMSS trend data for monitoring the effectiveness of their education systems in a global…

Stimulus-response (S-R) episodes are formed whenever a response is executed in close temporal proximity to a stimulus. Subsequent stimulus repetition will retrieve the episode from memory, reactivating the previous response. Whereas many research findings attest to the flexibility of representing stimulus features, only little is known about the…

In this paper we report the results of an experiment designed to test the hypothesis that when faced with a question involving the inverse direction of a reversible mathematical process, students solve a multiple-choice version by verifying the answers presented to them by the direct method, not by undertaking the actual inverse calculation.…

Automatic coding of short text responses opens new doors in assessment. We implemented and integrated baseline methods of natural language processing and statistical modelling by means of software components that are available under open licenses. The accuracy of automatic text coding is demonstrated by using data collected in the "Programme…

Examining the testing processes, as well as the scores, is needed for a complete understanding of validity and fairness of computer-based assessments. Examinees' rapid-guessing and insufficient familiarity with computers have been found to be major issues that weaken the validity arguments of scores. This study has three goals: (a) improving…

Humans are characterized by an especially highly developed ability to use instructions to prepare toward upcoming events; yet, it is unclear just how powerful instructions can be. Although prior work provides evidence that instructions can be sufficiently powerful to proactively program working memory to execute stimulus-response (S-R)…

Automated writing evaluation (AWE) systems can provide immediate computer-generated quantitative assessments and qualitative diagnostic feedback on an enormous number of submitted essays. However, limited research attention has been paid to locally designed AWE systems used in English as a foreign language (EFL) classroom contexts. This study…