Classroom observations increasingly inform high-stakes decisions and research in education, including the allocation of school funding and the evaluation of school-based interventions. However, trends in rater scoring tendencies over time may undermine the reliability of classroom observations. Accordingly, the present investigations, grounded in…

National attention on whole-of-school approaches to decrease children's sedentary behavior and increase physical activity includes movement integration (MI) in classrooms. The purpose of this study was to describe instrument development, reliability, and validity of the System for Observing Student Movement in Academic Routines and Transitions…

This article offers a way to measure agreement in participatory mapping. We asked subject matter experts (SMEs) to draw where Sudanese ethnic groups were located on a map. We then used an eigenanalysis approach to determine whether SMEs agreed on the location of ethnic groups. We used minimum residual factor analysis to assess the extent of…

The purpose of the present study was to determine the reliability of a 5km run test on a motorized treadmill. Over three consecutive weeks, 12 well-trained runners completed three 5km time trials on a treadmill following a standardized warm-up. Runners were partially-blinded to their running speed and distance covered. Total time to complete the…

The challenge of "educational" assessments--assessments that advance the purposes of learning and instruction--is to provide useful information regarding students' progress towards the goals of instruction in ways that are reliable and not idiosyncratic. In this commentary, the author indicates that the challenges are actually more…

In response to an argument by Baird, Andrich, Hopfenbeck and Stobart (2017), Michael Kane states that there needs to be a better fit between educational assessment and learning theory. In line with this goal, Kane will examine how psychometric constraints might be loosened by relaxing some psychometric "rules" in some assessment…

Design-based case studies address research questions that involve instructional innovations within a bounded system. This blend of case study and design-based research provides a systematic approach to examining instructional innovations that are bounded by perspective, context, and time. Design-based case studies provide a framework for engaging…

Although peer nomination measures have been used by researchers for nearly a century, common methodological practices and rules of thumb (e.g., which variables to measure; use of limited vs. unlimited nomination methods) have continued to develop in recent decades. At the same time, other key aspects of the basic nomination procedure (e.g.,…

There are two primary means to prevent the abuse of collegiality and transform it into a shield to protect the most vulnerable. First, colleges and universities should follow the examples of their peers by developing clear definitions of what types of behavior constitute collegiality and what types of activity are protected as academic freedom or…

Automated scoring systems are typically evaluated by comparing the performance of a single automated rater item-by-item to human raters. This presents a challenge when the performance of multiple raters needs to be compared across multiple items. Rankings could depend on specifics of the ranking procedure; observed differences could be due to…

There is an increasing demand for assessments that can provide more fine-grained information about examinees. In response to the demand, diagnostic measurement provides students with feedback on their strengths and weaknesses on specific skills by classifying them into mastery or nonmastery attribute categories. These attributes often form a…

Despite the expansion of early intervention approaches for young children with ASD, investigators have struggled to identify measures capable of assessing social communication change in response to these interventions. Addressing recent calls for efficient, sensitive, and reliable social communication measures, the current paper outlines the…

In this article I provide an overview of the methodological issues involved in measuring constructs relevant to character development and education. I begin with a nontechnical overview of the 3 fundamental psychometric properties of measurement: reliability, validity, and equivalence. Developing and evaluating measures to ensure evidence of all 3…

Bangladesh, home to one of the world's largest and most diverse education sectors, has witnessed rapid growth in its private education sector in recent decades. The majority of this growth has been within the low-fee private school (LFPS) sector, which now accounts for more than 25 percent of total school enrollment. However, the reliability of…

This paper examines the development and validation of the Survey of Climate for Instructional Improvement (SCII). SCII has 30-item Likert-scale items with 1 open response question and is designed to measure five aspects of organizational climate in postsecondary settings. The goal of this paper is to describe: (a) our development process, (b)…