In December 2022, federal, state and territory education ministers released a National Teacher Workforce Action Plan to improve teacher supply and retention in the profession. While recognising the important role that Highly Accomplished and Lead Teacher (HALT) certification could play, the plan also recognised that the current approach to…

Resistant behaviors in multicultural courses are a common dilemma within counselor education. On the basis of psychological reactance theory (J. W. Brehm, 1966), the authors created a multicultural training reactance scale to measure the construct. Exploratory factor analysis and initial reliability and validity data are presented, followed by…

This methodological paper examines current conceptions of reliability in chemistry education research (CER) and provides recommendations for moving beyond the current reliance on reporting coefficient alpha (a) as reliability evidence without regard to its appropriateness for the research context. To help foster a better understanding of…

Dynamic Measurement Modeling (DMM) is a psychometric paradigm that uses longitudinal data to estimate students' capacity to learn over the course of an educational program (i.e., growth scores). Here, we provide justification for this approach in health professions education and demonstrate its proof of concept with three time-points of USMLE Step…

In this article I provide an overview of the methodological issues involved in measuring constructs relevant to character development and education. I begin with a nontechnical overview of the 3 fundamental psychometric properties of measurement: reliability, validity, and equivalence. Developing and evaluating measures to ensure evidence of all 3…

Assessment is fundamental to school psychology, but its purpose has shifted from making predictions about children to improving outcomes for children. This commentary on the special issue focuses on screening and progress-monitoring decisions that can be used to solve student problems. We outline several psychometric and practical issues that…

Over the last five years, the U.S. Department of Education (USED) is requiring states to provide more and better evidence supporting the quality and validity of their assessment and accountability systems. Notably, states are asked to explain more fully how the proposed design of their systems will provide accurate and reliable information that…

This paper provides a conceptual, empirical, and practical guide for estimating ordinal reliability coefficients for ordinal item response data (also referred to as Likert, Likert-type, ordered categorical, or rating scale item responses). Conventionally, reliability coefficients, such as Cronbach's alpha, are calculated using a Pearson…

This article shows how interfactor correlation is affected by error correlations. Theoretical and practical justifications for error correlations are given, and a new equivalence class of models is presented to explain the relationship between interfactor correlation and error correlations. The class allows simple, parsimonious modeling of error…

Composition assessment scholars have exhibited uneasiness with the language of norming grounded in distaste for the psychometric assumption that achievement of consensus in a communal assessment setting is desirable even at the cost of individual pedagogical values. Responding to the problems of a "reliability" defined by homogenous agreement,…

Recently, student outcomes have bubbled to the top of debates about how to evaluate teaching in community and liberal arts colleges, universities, and professional schools, but even more international attention has been riveted on how outcomes are being used to evaluate teachers and administrators K-12 (Harris, 2012; Rowen & Raudenbush, 2016;…

Everyone who runs a youth program believes in their hearts that their program helps kids, but in their heads, they know they need convincing data to prove it. This guide--updated from 2011--is here to help them get the data they need. The guide addresses a common problem throughout the youth field: Out-of-school time (OST) programs can help youth…

The rise in the prevalence of autism creates a need for a reliable and valid measure of attitudes towards autism. The current study describes the development of a brief 16-item measure of Societal Attitudes towards Autism (SATA) that exhibits sound psychometric properties and has a demonstrated ability to discriminate between expert and general…

In the scholarship of evaluating student assessment, no group does it as well as the Indiana University scholars engaged in the National Survey of Student Engagement (NSSE) and Community College Survey of Student Engagement (CCSSE). More than 1,400 institutions employ the NSSE as a tool for better understanding their students' involvement,…

As pointed out by Sijtsma ("in press"), coefficient alpha is inappropriate as a single summary of the internal consistency of a composite score. Better estimators of internal consistency are available. In addition to those mentioned by Sijtsma, an old dimension-free coefficient and structural equation model based coefficients are…