This paper presents the concepts and observations in the author's keynote address at the May 2019 "Opportunity versus Challenge: Exploring Usage of Log-File and Process Data in International Large-Scale Assessments" conference in Dublin, Ireland. This paper recaps briefly some key points that emerged at the December 2018 ETS symposium on…

School systems across the country are transitioning from paper-based testing (PBT) to computer-based testing (CBT). As this technological shift occurs, more research is necessary to understand the practical and performance implications of administering CBTs. Currently, there is a paucity of research using CBTs to examine working memory (WM)…

Many universities rely on data gathered from tests that are low stakes for examinees but high stakes for the various programs being assessed. Given the lack of consequences associated with many collegiate assessments, the construct-irrelevant variance introduced by unmotivated students is potentially a serious threat to the validity of the…

Fifty children 7 years of age (29 girls, 21 boys), 53 children 10 years of age (29 girls, 24 boys), and 36 young adults (19 women, 17 men) performed a computerized event-based prospective memory task. All 3 groups differed significantly in prospective memory performance, with adults showing the best performance and with 7-year-olds showing the…

This study investigated the reading behavior of 15-year-old students while reading texts and answering corresponding multiple-choice questions. The availability of the texts during question answering was manipulated experimentally. Allocation of resources to several cognitive processes at the word, sentence, and text level was measured by…

The Wisconsin Card Sorting Test (WCST) assesses executive and frontal lobe function and can be administered manually or by computer. Despite the widespread application of the 2 versions, the psychometric equivalence of their scores has rarely been evaluated and only a limited set of criteria has been considered. The present experimental study (N =…

Information Integration Theory (IIT) is concerned with how people combine information into an overall judgment. A method is hereby presented to perform Functional Measurement (FM) experiments, the methodological counterpart of IIT, on the Web. In a comparison of Web-based FM experiments, face-to-face experiments, and computer-based experiments in…

The aim of the present study was to assess mental flexibility and set maintenance of a group of individuals with Asperger syndrome (AS) or high-functioning autism (HFA) (N = 13; mean age 16,4), as compared with a matched group of typically developing children and adolescents (N = 13; mean age 15,6) on the computerized version of the Wisconsin Card…

The ability to select an appropriate motor response by resolving competition among alternative responses plays a major role in cognitive performance. fMRI studies suggest that the development of this skill is related to the maturation of the frontal cortex that underlies the improvement of motor inhibition abilities. However, fMRI cannot…

With the advent of the Internet, delivered assessment applications are likely to make a difference in clinical psychology and in health care in general. The purpose of the present paper was to present an overview of the authors' experience regarding Internet administration of diagnostic interviews, questionnaires, and information processing tests.…

One of the central challenges in mastering English is becoming sensitive to consistency from spelling to sound (i.e. phonological consistency) and from sound to spelling (i.e. orthographic consistency). Using functional magnetic resonance imaging (fMRI), we examined the neural correlates of consistency in 9-15-year-old Normal and Impaired Readers…

In current psychological research, the analysis of data from computer-based assessments or experiments is often confined to accuracy scores. Response times, although being an important source of additional information, are either neglected or analyzed separately. In this article, a new model is developed that allows the simultaneous analysis of…

This paper investigates how monitoring the time spent on a question in a test of basic mathematics skills can provide insights into learning processes, the quality of test takers' knowledge, and cognitive demands and performance of test items that otherwise would remain undiscovered if the usual test outcome of accuracy only format…

Most of the emphasis on mining online assessment logs has been to identify content-specific errors. However, the pattern of general "consistency" is domain independent, strongly related to performance, and can itself be a target of educational data mining. We demonstrate that simple consistency indicators are related to student outcomes,…

Ubiquitous computing and mobile technologies provide a new perspective for designing innovative outdoor learning experiences. The purpose of this study is to propose a formative assessment design for integrating PDAs into ecology observations. Three learning activities were conducted in this study. An action research approach was applied to…