Scores estimated from multidimensional item response theory (IRT) models are not necessarily comparable across dimensions. In this article, the concept of aligned dimensions is formalized in the context of Rasch models, and two methods are described--delta dimensional alignment (DDA) and logistic regression alignment (LRA)--to transform estimated…

Researchers in many disciplines work with ranking data. This data type is unique in that it is often deterministic in nature (the ranks of items "k"-1 determine the rank of item "k"), and the difference in a pair of rank scores separated by "k" units is equivalent regardless of the actual values of the two ranks in…

Mokken scale analysis (MSA) is a probabilistic-nonparametric approach to item response theory (IRT) that can be used to evaluate fundamental measurement properties with less strict assumptions than parametric IRT models. This instructional module provides an introduction to MSA as a probabilistic-nonparametric framework in which to explore…

Massive Open Online Courses (MOOCs) are becoming an increasingly popular choice for education but, to reach their full extent, they require the resolution of new issues like assessing students at scale. A feasible approach to tackle this problem is peer assessment, in which students also play the role of assessor for assignments submitted by…

Assessing complex constructs such as those discussed under the umbrella of 21st century constructs highlights the need for a principled assessment design and validation approach. In our discussion, we made a case for three considerations: (a) taking construct complexity into account across various stages of assessment development such as the…

The problem of building test collections is central to the development of information retrieval systems such as search engines. The primary use of test collections is the evaluation of IR systems. The widely employed "Cranfield paradigm" dictates that the information relevant to a topic be encoded at the level of documents, therefore…

We examined predictions derived from Valentine's (1991) Multidimensional Space (MDS) framework for own- and other-race face processing. A set of 20 computerized faces was generated from a single prototype. Each face was saved as Black and White, changing only skin tone, such that structurally identical faces were represented in both race…

A simulation study compared the performance of robust normal theory maximum likelihood (ML) and robust categorical least squares (cat-LS) methodology for estimating confirmatory factor analysis models with ordinal variables. Data were generated from 2 models with 2-7 categories, 4 sample sizes, 2 latent distributions, and 5 patterns of category…

The primary users of the current college ranking systems do not seem to be high-school students and families, but college presidents, board members, and development officers. As structured, the commercial ranking systems imply a precision that is not corroborated by research on what matters in college, nor can college quality be accurately summed…

A disconnect exists between principal leadership expectations and the actual practice of supervision and evaluation of principals. Increased principal responsibilities and rigorous standards require that a new approach to principal evaluation be considered. The authors provide a framework for a multi-dimensional approach to principal supervision…

This study explores the utility of a dual-factor model of mental health in which the concepts of mental illness and mental wellness are integrated. Life satisfaction, emotional symptoms, personal adjustment, and clinical symptoms were assessed with a sample of 240 college students. Participants were organized into four groups based on levels of…

Multiplicative interaction models, such as Goodman's (1981) RC(M) association models, can be a useful tool for analyzing the content of interaction effects. However, most models for interaction effects are suitable only for data sets with two or three predictor variables. Here, we discuss an optimal scaling model for analyzing the content of…

The purpose of this brief is to understand the Illinois College and Career Readiness (CCR) Act in light of David Conley's college readiness model. Although not mentioned specifically by the Illinois statute, evaluation results gathered by the Office of Community College Research and Leadership (OCCRL) show alignment between a number of programs…

Judging from the recent surge in research on outcomes assessment, many study abroad offices rely on quantitative surveys and measures to collect student outcomes data. This paper presents a multidimensional, qualitative approach to data collection. This approach makes qualitative data easy to collect and encourages students to reflect and…

In this article, the relationship between two alternative methods for the analysis of multivariate categorical data is systematically explored. It is shown that the person score of the first dimension of classical optimal scaling correlates strongly with the latent variable for the two-parameter item response theory (IRT) model. Next, under the…