The study focuses on assessing the proficiency levels of higher education students, specifically the physics achievement test (PHY 101) at the National Open University of Nigeria (NOUN). This test, like others, evaluates various aspects of knowledge and skills simultaneously. However, relying on traditional models for such tests can result in…

This study reports on an Evidence Centered Design (ECD) project in the Netherlands, involving the theory exam for prospective car drivers. In particular, we illustrate how cognitive load theory, task-analysis, response process models, and explanatory item-response theory can be used to systematically develop and refine task models. Based on a…

Certain item features can be used to explain the difference between the difficulties of items. These item features can relate to the steps necessary to solve the problems or ways in which a increased understanding of material is acquired. This study will examine the relationship between item difficulty and the process skills associated with these…

This study evaluates a 12-item instrument for subjective measurement of mental load (ML) and mental effort (ME) by analysing different sources of validity evidence. The findings of an expert judgement (N = 8) provide "evidence based on test content" that the formulation of the items corresponds to the meaning of ML and ME. An empirical…

The study aims at developing and exploring a novel video-based assessment that captures classroom management expertise (CME) of teachers and for which statistical results are provided. CME measurement is conceptualized by using four video clips that refer to typical classroom management situations in which teachers are heavily challenged…

Reading for learning frequently requires integrating text and picture information into coherent knowledge structures. This article presents an experimental study aimed at analyzing the strategies used by students for integrating text and picture information. Four combinations of texts and pictures (text-picture units) were selected from textbooks…

Strategic recollection refers to control processes that allow the retrieval of information that is relevant for a specific situation. These processes can be studied in memory exclusion tasks, which require the retrieval of particular kinds of episodic information. In the current study, we investigated strategic recollection in reality monitoring…

The Latin Square Task (LST) was developed by Birney, Halford, and Andrews [Birney, D. P., Halford, G. S., & Andrews, G. (2006). Measuring the influence of cognitive complexity on relational reasoning: The development of the Latin Square Task. Educational and Psychological Measurement, 66, 146-171.] and represents a non-domain specific,…

The purpose of the present investigation was to identify the relationship among different indicators of uncertainty that lead to potential item misalignment. The item-based indicators included ratings of ambiguity and cognitive complexity. The student-based indicators included (a) frequency of cognitive monitoring per item, (b) levels of…

In an effort to improve the ways in which words are learned, the Johnson O'Connor Research Foundation (JOCRF) is attempting to determine the difficulty level of all non-technical words in the English language. This item banking project entails: (1) identifying words that should be calibrated; (2) writing a test item for each word; (3) testing the…

An approach is described that reveals the hierarchical test structure (HTS) based on the cognitive demands of the test items, and conducts a linear trait modeling by using the HST elements as item difficulty components. This approach, referred to as the Hierarchical Latent Trait Approach (HLTA), employs an algorithm that allows all test items to…

Simple arithmetic word problems are often featured in elementary school education. One type of problem, "compare with unknown reference set," ranks among the most difficult to solve. Differences in item difficulty for compare problems with unknown reference set are observed depending on the direction of the relational statement (more than vs. less…

Recent assessment research joining cognitive psychology and psychometric theory has introduced a new technology, item generation. In algorithmic item generation, items are systematically created based on specific combinations of features that underlie the processing required to correctly solve a problem. Reading comprehension items have been more…

Context effects in test taking were explored, paying attention to the psychological processes that occur during test taking, and modeling context effects for each individual at the item block level. A sample of 279 high school students (140 females and 139 males) was chosen to yield adequate power for detecting interactions. Reading test forms…

The 4th International Conference on Educational Data Mining (EDM 2011) brings together researchers from computer science, education, psychology, psychometrics, and statistics to analyze large datasets to answer educational research questions. The conference, held in Eindhoven, The Netherlands, July 6-9, 2011, follows the three previous editions…