Coefficient alpha reliability persists as the most common reliability coefficient reported in research. The assumptions for its use are, however, not well-understood. The current paper challenges the commonly used expressions of coefficient alpha and argues that while these expressions are correct when estimating reliability for summed scores,…

The multistage testing (MST) design has been gaining attention and popularity in educational assessments. For testing programs that have small test-taker samples, it is challenging to calibrate new items to replenish the item pool. In the current research, we used the item pools from an operational MST program to illustrate how research studies…

We developed and validated scores on the Lifestyle Practices and Health Consciousness Inventory (LPHCI)-2: Brief Version, a short form for measuring global wellness (mental and physical health). Tests of internal structure (EFA, CFA, and higher-order CFA) as well as convergent validity supported the psychometric properties of LPHCI-2: Brief…

Purpose: The study was aimed at measuring farmers' helpline services quality in India using a standardized multi-factor scale (HELPQUAL) developed as part of this study. Design/methodology/approach: The present study is based on 360 farmers' and 45 experts' responses gathered using telephonic interviews and mailed questionnaires during the year…

In this Brief Report, we describe an evaluation of and revisions to a rubric adapted from the Jacobs et al.'s (1981) ESL COMPOSITION PROFILE, with four rubric categories and 20-point rating scales, in the context of an intensive English program writing placement test. Analysis of 4 years of rating data (2016-2021, including 434 essays) using…

Raven's Standard Progressive Matrices (Raven 1941) is a widely used 60-item long measure of general mental ability. It was recently suggested that, for situations where taking this test is too time consuming, a shorter version, comprised of only the last series of the Standard Progressive Matrices (Myszkowski and Storme 2018) could be used, while…

Much of what we know about how children develop is based on survey data. In order to estimate growth across time and, thereby, better understand that development, short survey scales are typically administered at repeated timepoints. Before estimating growth, those repeated measures must be put onto the same scale. Yet, little research examines…

The purpose of this article was two-fold (1) provide an overview of the commonly reported and under-reported absolute agreement indices in the kinesiology literature for continuous data; and (2) present examples of these indices for hypothetical data along with recommendations for future use. It is recommended that three types of information be…

A distinction is made between scores as measures of a construct and predictions of a criterion or outcome variable. The interpretation attached to predictions of criteria, such as job performance or college grade point average (GPA), differs from that attached to scores that are measures of a construct, such as reading proficiency or knowledge…

The purpose of this study was to use multidimensional scaling (MDS) to investigate relations among scores from the standardization sample of the Wechsler Intelligence Scale for Children--Fifth edition (WISC-V; Wechsler, 2014). Nonmetric two-dimensional MDS maps were selected for interpretation. The most cognitively complex subtests and indexes…

Prior to August 2015, the National Board of Medical Examiners' (NBME) clinical science subject examination scores were reported as a scaled score. However, the scaled scores had some undesirable properties. Medical school faculty often interpreted the scaled scores as though they were percent correct scores despite cautions that this…

The OECD's Programme for International Student Assessment (PISA) has become one of the key studies for evidence-based education policymaking across the globe. PISA has however received a lot of methodological criticism, including how the test scores are created. The aim of this paper is to investigate the so-called 'conditioning model', where…

Linking score scales across different tests is considered speculative and fraught, even at the aggregate level (Feuer et al., 1999; Thissen, 2007). We introduce and illustrate validation methods for aggregate linkages, using the challenge of linking U.S. school district average test scores across states as a motivating example. We show that…

This article describes the development and validation of an Innovation Attitude Survey (IAS) composed of 16 Likert-type items selected to measure middle school students' attitudes toward innovation and leadership in the advancement of new ideas. The goal of developing the IAS was to identify desirable dispositions that may be related to future…

The development of international surveys on children's learning like PISA, PIRLS and TIMSS--delivering comparable achievement measures across educational systems--has revealed large cross-country variability in average performance and in the degree of inequality across social groups. A key question is whether and how institutional differences…