Back random responding (BRR) behavior is one of the commonly observed careless response behaviors. Accurately detecting BRR behavior can improve test validities. Yu and Cheng (2019) showed that the change point analysis (CPA) procedure based on weighted residual (CPA-WR) performed well in detecting BRR. Compared with the CPA procedure, the…

Educators commonly use school climate survey scores to inform and evaluate interventions for equitably improving learning and reducing educational disparities. Unfortunately, validity evidence to support these (and other) score uses often falls short. In response, Whitehouse et al. proposed a collaborative, two-part validity testing framework for…

Assessment in mathematics is assumed to provide credible and important information about what students know and can do. In this paper we focus on large scale tests and question whether mathematics assessment is essentially gender neutral. We consider aspects of test validity and discuss issues of terminology related to gender and mathematics. In…

Measurement invariance is a prerequisite for comparing measurement scores from different groups. In medical education, multi-source feedback (MSF) is utilized to assess core competencies, including the professionalism. However, little attention has been paid to the measurement invariance of assessment instruments; that is, whether an instrument…

Although there is a strong legislative base and perceived efficacy for multidisciplinary team decision making, limited evidence supports its effectiveness or consistency of implementation in practice. In recent research, we used the Decision Observation, Recording, and Analysis (DORA) tool to document activities and adult behaviors during positive…

This study assesses the psychometric properties of a newly tested self-report assessment tool for media literacy, based on the twelve new media literacy skills (NMLs) developed by Jenkins et al. (2006). The sample (N = 327) consisted of normal volunteers who completed a comprehensive online survey that measured their NML skills, media exposure,…

International large-scale assessments of achievement often have a large degree of differential item functioning (DIF) between countries, which can threaten score equivalence and reduce the validity of inferences based on comparisons of group performances. It is important to understand potential sources of DIF to improve the validity of future…

Beginning with a reference to living in a time of both uncertainty and opportunity, this article presents a discussion of key areas where shared understanding is needed if we are to successfully realize the design and use of high quality, valid assessments of science. The key areas discussed are: (1) assessment purpose and use, (2) the nature of…

The analysis of response inconsistency is a crucial aspect of intellectual and clinical psychological assessment. Erratic patterns of failures and successes across and within particular domains qualify the measurement of intellectual potential and functioning. Although the interpretation of intertest scatter (inconsistencies between subtest…

The need for rigorously developed measures of preservice teacher traits regarding technology integration training has been acknowledged (Kay 2006), but such instruments are still extremely rare. The Technology Integration Confidence Scale (TICS) represents one such measure, but past analyses of its functioning have been limited by sample size and…

This study assessed the convergent and divergent validity of the Devereux Student Strengths Assessment (DESSA; LeBuffe, Shapiro, & Naglieri, 2008). A total of 227 (n = 94 teachers, n = 133 parents) raters completed the DESSA, in addition to the Behavioral and Emotional Rating Scales-2 (BERS-2; Epstein, 2004) and/or the Behavior Assessment…

This article develops a model for longitudinal student achievement data designed to estimate heterogeneity in teacher effects across students of different achievement levels. The model specifies interactions between teacher effects and students' predicted scores on a test, estimating both average effects of individual teachers and interaction…

The importance of student assessment, particularly for summative purposes, has increased greatly over the past thirty years. At the same time, emphasis on including all students in assessment programs has also increased. Assessment programs, whether they are large-scale, district-based, or teacher developed, have traditionally attempted to assess…

In response to Lissitz and Samuelsen (2007), the author reconstructs the historical arguments for the more comprehensive unitary concept of validity and the principles of scientific inquiry underlying it. Her response is organized in terms of four questions: (a) How did validity in educational measurement come to be conceptualized as unitary, and…

The five papers in this special issue of the "Journal of Applied Testing Technology" address fundamental issues of validity when tests are modified or accommodations are provided to English Language Learners (ELL) or students with disabilities. Three papers employed differential item functioning (DIF) and factor analysis and found the…