This study adopted a multilevel modeling (MLM) approach to examine the contribution of rater and essay factors to variability in ESL essay holistic scores. Previous research aiming to explain variability in essay holistic scores has focused on either rater or essay factors. The few studies that have examined the contribution of more than one…

Analysis of children's productions of oral narratives provides a rich description of children's oral language skills. However, measures of narrative organization can be directly affected by both developmental and task-based performance constraints which can make a measure insensitive and inappropriate for a particular population and/or sampling…

The purpose of this integrative literature review is to summarize research on concept mapping and to offer ideas on how concept mapping can facilitate practice, research, and theory development within human resource development. In this review, more than 300 articles, written in both English and Spanish, presented at two different concept mapping…

A nonparametric tree classification procedure is used to detect differential item functioning for items that are dichotomously scored. Classification trees are shown to be an alternative procedure to detect differential item functioning other than the use of traditional Mantel-Haenszel and logistic regression analysis. A nonparametric…

This article reviews the "Developmental Reading Assessment--Second Edition" (DRA2), a teacher-administered assessment that identifies students' instructional level, along with their strengths and weaknesses in reading. Once teachers calculate and interpret scores, the data can purportedly be used to inform, and possibly individualize,…

Standard setting is a systematic process that uses a combination of judgmental and empirical procedures to make recommendations about where on the score continuum "cut scores" should be placed. Cut scores divide the score scale into categories consistent with the descriptions of student performance associated with multiple levels of achievement.…

This longitudinal study examines the biliteracy results of Spanish-English emerging bilingual students who participated in a K-5 paired literacy model in a large school district in Oregon. Spanish and English reading and writing data show longitudinal gains in students' biliterate development, demonstrating the potential of the model in developing…

This report summarizes the complex research literature on quantitative methods for assessing how impacts of educational interventions on instructional practices and student learning differ across students, educators, and schools. It also provides technical guidance about the use and interpretation of these methods. The research topics addressed…

Exploring the critical role of school choice in the future of education reform, the Education Choice and Competition Index (ECCI) is an interactive web application that scores large school districts based on thirteen categories of policy and practice relevant to choice. The intent of the ECCI is to: create public awareness of the differences among…

Even when the scoring of an examination is based on item response theory (IRT), standard-setting methods seldom use this information directly when determining the minimum passing score (MPS) for an examination from an Angoff-based standard-setting study. Often, when IRT scoring is used, the MPS value for a test is converted to an IRT-based theta…

Essential for the validity of the judgments in a standard-setting study is that they follow the implicit task assumptions. In the Angoff method, judgments are assumed to be inversely related to the difficulty of the items; contrasting-groups judgments are assumed to be positively related to the ability of the students. In the present study,…

Background: The Digital Forms of Assessment project is a three-year Australian Research Council Linkage research project being conducted by Edith Cowan University's Centre for Schooling and Learning Technologies (CSaLT) in partnership with the Curriculum Council of Western Australia. The project is investigating the potential for a digital format…

A total of 47 gifted and nongifted African American and Euro-American elementary students were rated by their teachers on a multidimensional instrument developed to minimize language considerations and to rely on local norms (Universal Multiple Abilities Scales [UMAS; McCallum & Bracken, 2012a]). Results from two factorial MANOVAs revealed no…

This article develops a validity argument approach for use on observation protocols currently used to assess teacher quality for high-stakes personnel and professional development decisions. After defining the teaching quality domain, we articulate an interpretive argument for observation protocols. To illustrate the types of evidence that might…

The Draw an Engineer Test (DAET) is a common measure of students' perceptions of engineers. The coding systems currently used for K-12 research are general rubrics or checklists to capture the images presented in the drawing, which leave out some of the richness of students' perceptions, currently only captured with an accompanying student…