The multiple-choice (MC) item format has been widely used in educational assessments across diverse content domains. MC items purportedly allow for collecting richer diagnostic information. The effectiveness and economy of administering MC items may have further contributed to their popularity not just in educational assessment. The MC item format…

This paper presents the concepts and observations in the author's keynote address at the May 2019 "Opportunity versus Challenge: Exploring Usage of Log-File and Process Data in International Large-Scale Assessments" conference in Dublin, Ireland. This paper recaps briefly some key points that emerged at the December 2018 ETS symposium on…

This book is a testament to the increasing role and importance of multiple source use in everyday and academic literacy activities in the 21st century. How should we introduce the topic of multiple sources here? For most readers, we need not, because this chapter is not their first literacy stop in the volume, so the topic has been adequately…

A family of learning models that integrates a cognitive diagnostic model and a higher-order, hidden Markov model in one framework is proposed. This new framework includes covariates to model skill transition in the learning environment. A Bayesian formulation is adopted to estimate parameters from a learning model. The developed methods are…

Cognitive diagnosis models provide profile information about a set of latent binary attributes, whereas item response models yield a summary report on a latent continuous trait. To utilize the advantages of both models, higher order cognitive diagnosis models were developed in which information about both latent binary attributes and latent…

"Classroom Assessment and Educational Measurement" explores the ways in which the theory and practice of both educational measurement and the assessment of student learning in classroom settings mutually inform one another. Chapters by assessment and measurement experts consider the nature of classroom assessment information, from…

This study evaluated a new measure for analyzing the process of children's problem solving in a series completion task. This measure focused on a process that we entitled the "Grouping of Answer Pieces" (GAP) that was employed to provide information on problem representation and restructuring. The task was conducted using an electronic…

The number of students engaged in Massive Open Online Courses (MOOCs) is increasing rapidly. Due to the autonomy of students in this type of education, students in MOOCs are required to regulate their learning to a greater extent than students in traditional, face-to-face education. However, there is no questionnaire available suited for this…

Cognitive scientists investigate mental models (how humans organize and structure knowledge in their minds) so as to understand human understanding of and interactions with the world. Cognitive and mental model research is concerned with internal conceptual systems that are not easily or directly observable. The goal of this research was to…

Cognitive ability tests are widely used in organizations around the world because they have high predictive validity in selection contexts. Although these tests typically measure several subdomains, testing is usually carried out for a single subdomain at a time. This can be ineffective when the subdomains assessed are highly correlated. This…

Much recent psychometric literature has focused on cognitive diagnosis models (CDMs), a promising class of instruments used to measure the strengths and weaknesses of examinees. This article introduces a genetic algorithm to perform automated test assembly alongside CDMs. The algorithm is flexible in that it can be applied whether the goal is to…

Effective and efficient measurement of the development of skill and knowledge, especially in domains of human activity that involve complex and challenging problems, is important with regard to workplace and academic performance. However, there has been little progress in the area of practical measurement and assessment, due in part to the lack of…

Marked gender differences in three-dimensional mental rotation have been broadly reported in the literature in the last few decades. Various theoretical models and accounts were used to explain the observed differences. Within the framework of linking item design features of mental rotation tasks to cognitive component processes associated with…

This paper defines Bayesian network models and examines their applications to IRT-based cognitive diagnostic modeling. These models are especially suited to building inference engines designed to be synchronous with the finer grained student models that arise in skills diagnostic assessment. Aspects of the theory and use of Bayesian network models…

Though the most widely used "Intelligence" tests have changed little in the last half century, the author predicts a variety of new influences of future intelligence testing, ranging from the ways we conceptualize intelligence, to the manner in which we pose problems to assess it. (Author/JAZ)