This article is a written version of the Presidential Address I gave at the annual meeting of the National Council on Measurement in Education (NCME) in April 2017. It is a call to NCME members (and others who read this, of course) to rebalance their focus so that classroom assessments are seen as being at least as important as large-scale…

This study proposes a structured constructs model (SCM) to examine measurement in the context of a multidimensional learning progression (LP). The LP is assumed to have features that go beyond a typical multidimentional IRT model, in that there are hypothesized to be certain cross-dimensional linkages that correspond to requirements between the…

In this paper, we describe a new method we have developed for setting cut scores between levels of a test. We outline the wide variety of potential methods that have been used for such a process, and emphasize the need for a coherent conceptual framework under which the variety of methods could be understood. We then describe our particular…

In this rejoinder, the authors provide their thoughts on each of the commentaries of the seven respondents to their article. They find that the response of Kyngdon differs markedly from the others in questioning some basic elements of the methods of analysis that they propose for the construction of a "road map." The authors emphasize that they…

This article describes some of the underlying conceptualizations that have gone into the work of the BEAR Center in the development of learning progressions. The core of all of these developments has been the construct map, which is the first building block in the BEAR Assessment System (BAS). After introducing the concept of a learning…

The overall aim of this article is to analyze the relationships between the roles of assessment in pedagogy, the interactions between curriculum assessment and pedagogy, and the study of pupils' progression in learning. It is argued that well-grounded evidence of pupils' progressions in learning is crucial to the work of teachers, so that a method…

An approach called generalizability in item response modeling (GIRM) is introduced in this article. The GIRM approach essentially incorporates the sampling model of generalizability theory (GT) into the scaling model of item response theory (IRT) by making distributional assumptions about the relevant measurement facets. By specifying a random…

The Rasch testlet model for both dichotomous and polytomous items in testlet-based tests is proposed. It can be viewed as a special case of the multidimensional random coefficients multinomial logit model (MRCMLM). Therefore, the estimation procedures for the MRCMLM can be directly applied. Simulations were conducted to examine parameter recovery…

When measuring participant-reported attitudes and outcomes in the behavioral sciences, there are many instances when the common measurement assumption of unidimensionality does not hold. In these cases, the application of a multidimensional measurement model is both technically appropriate and potentially advantageous in substance. In this paper,…

Another Rasch approach to the measurement of change, the multidimensional random coefficient multinomial logit model (MRCML), is proposed. The MRCML model can be applied to polytomous items and the investigation of variations in item difficulties. Some simulation studies demonstrate good parameter recovery for the MRCML model under various testing…

An important, sometimes controversial feature of all psychological phenomena is whether they are categorical or dimensional. A conceptual and psychometric framework is described for distinguishing whether the latent structure behind manifest categories (e.g., psychiatric diagnoses, attitude groups, or stages of development) is category-like or…

Studied the usefulness of the Multidimensional Random Coefficient Multinomial Logit model using a collection of embedded assessments administered to 541 students in middle school science classes. Findings illustrate the usefulness and good fit of the multidimensional approach in contrast to a unidimensional approach. (SLD)

The ordered partition model is described as an extension of the partial credit model, its relationship to other models is discussed, and two examples are presented. The model is designed for a measurement context in which categories of response to an item cannot be completely ordered. (SLD)

The emphasis of mastery learning instruction is unique in that concern is not only with raising the level of achievement but also with reducing the variability of achievement among the students instructed. The Glass effect size estimator, presently used an an index to measure the effectiveness of mastery learning experiments, is examined relative…

This study presents a procedure for detecting differential item functioning (DIF) for dichotomous and polytomous items in testlet-based tests, whereby DIF is taken into account by adding DIF parameters into the Rasch testlet model. Simulations were conducted to assess recovery of the DIF and other parameters. Two independent variables, test type…