The ABC Quality program, South Carolina's Quality Rating and Improvement System (QRIS), is a statewide system designed to improve the quality of early care and education services. Historically, two observation instruments are used for centers that elect to participate in the state's voluntary QRIS depending on the level of quality for which they…

The long-term goal of the State Systemic Improvement Plan (SSIP) and other federal and state early intervention and early childhood education initiatives is improved child and family outcomes. States play a critical role in supporting practitioners in the use of evidence-based practices to improve child and family outcomes. When practitioners…

The purpose of this study is to further enhance the existing reliability and validity evidence related to the School Implementation Scale (SIS) and its use for evaluating the implementation of different integrated academic and behavior tiered Response-To-Intervention (RTI) models. Previous methodology conducted for validation of the instrument has…

Community-based education programs must demonstrate effectiveness to various funding sources. The pilot study reported here (funded by CYFAR, NIFA, USDA award #2008-41520-04810) had the goal of determining if state level programs with varied curriculum could use a common evaluation tool to demonstrate efficacy. Results in parenting and youth…

This article proposes that sampling design effects have potentially huge unrecognized impacts on the results reported by large-scale district and state assessments in the United States. When design effects are unrecognized and unaccounted for they lead to underestimating the sampling error in item and test statistics. Underestimating the sampling…

This study evaluated large-scale state writing assessments for the inclusion of motivational characteristics in the writing task and written prompt. We identified 6 motivational variables from the authentic activity literature: time allocation, audience specification, audience intimacy, definition of task, allowance for multiple perspectives, and…

State-level testing programs continue to grow, and the challenge of validation does not wane. Although more than a decade has passed since the 1999 Joint Standards for Educational and Psychological Testing set out a call for the organization of validity evidence into validity arguments, practical examples of such arguments are not readily…

Systematic evaluation is an essential tool for understanding program effectiveness. This article describes the pilot test of a statewide evaluation tool for the Supplemental Nutrition Assistance Program-Education (SNAP-Ed). A computer algorithm helped Community Nutrition Educators (CNEs) build surveys specific to their varied educational settings…

This study is an evaluation of the psychometric issues associated with estimating objective level scores, often referred to as "subscores." The article begins by introducing the concepts of reliability and validity for subscores from statewide achievement tests. These issues are discussed with reference to popular scaling techniques, classical…

As part of a Response to Intervention (RTI) model, curriculum-based measurement (CBM) is an evidence-based assessment system that uses screening and formative assessments to help guide instruction. The two primary purposes of this study are to examine the relations between the easyCBM benchmark measures and a statewide large-scale assessment and…

The relationships between ratings on the Idaho Alternate Assessment (IAA) for 116 students with significant disabilities and corresponding ratings for the same students on two norm-referenced teacher rating scales were examined to gain evidence about the validity of resulting IAA scores. To contextualize these findings, another group of 54…

The research for this study was undertaken to analyze the New York State 11th grade United States History Regents exams through conducting a content analysis of the types of multiple-choice questions asked in Part I of the tests with a particular emphasis on the variety of questions asked regarding women and race. Because these tests stand at the…

This report summarizes findings from a study of the psychometric properties of the resilience and youth development module, a key component of the Healthy Kids Survey. The study aims to improve resilience assessment and research so that educators can shape the school environment to promote academic resilience. The Healthy Kids Survey (HKS) is a…

At the heart of the No Child Left Behind Act (NCLB) is the call for all students to be "proficient" in reading and mathematics by 2014. Yet the law expects each state to define proficiency as it sees fit and design its own tests. This study investigated three research questions related to this policy: (1) How consistent are various…