The ABC Quality program, South Carolina's Quality Rating and Improvement System (QRIS), is a statewide system designed to improve the quality of early care and education services. Historically, two observation instruments are used for centers that elect to participate in the state's voluntary QRIS depending on the level of quality for which they…

The long-term goal of the State Systemic Improvement Plan (SSIP) and other federal and state early intervention and early childhood education initiatives is improved child and family outcomes. States play a critical role in supporting practitioners in the use of evidence-based practices to improve child and family outcomes. When practitioners…

The purpose of this study is to further enhance the existing reliability and validity evidence related to the School Implementation Scale (SIS) and its use for evaluating the implementation of different integrated academic and behavior tiered Response-To-Intervention (RTI) models. Previous methodology conducted for validation of the instrument has…

Community-based education programs must demonstrate effectiveness to various funding sources. The pilot study reported here (funded by CYFAR, NIFA, USDA award #2008-41520-04810) had the goal of determining if state level programs with varied curriculum could use a common evaluation tool to demonstrate efficacy. Results in parenting and youth…

This article proposes that sampling design effects have potentially huge unrecognized impacts on the results reported by large-scale district and state assessments in the United States. When design effects are unrecognized and unaccounted for they lead to underestimating the sampling error in item and test statistics. Underestimating the sampling…

This study evaluated large-scale state writing assessments for the inclusion of motivational characteristics in the writing task and written prompt. We identified 6 motivational variables from the authentic activity literature: time allocation, audience specification, audience intimacy, definition of task, allowance for multiple perspectives, and…

State-level testing programs continue to grow, and the challenge of validation does not wane. Although more than a decade has passed since the 1999 Joint Standards for Educational and Psychological Testing set out a call for the organization of validity evidence into validity arguments, practical examples of such arguments are not readily…

Systematic evaluation is an essential tool for understanding program effectiveness. This article describes the pilot test of a statewide evaluation tool for the Supplemental Nutrition Assistance Program-Education (SNAP-Ed). A computer algorithm helped Community Nutrition Educators (CNEs) build surveys specific to their varied educational settings…

This study is an evaluation of the psychometric issues associated with estimating objective level scores, often referred to as "subscores." The article begins by introducing the concepts of reliability and validity for subscores from statewide achievement tests. These issues are discussed with reference to popular scaling techniques, classical…

As part of a Response to Intervention (RTI) model, curriculum-based measurement (CBM) is an evidence-based assessment system that uses screening and formative assessments to help guide instruction. The two primary purposes of this study are to examine the relations between the easyCBM benchmark measures and a statewide large-scale assessment and…

It is argued that publishers of achievement tests, especially those who publish tests intended for use in many parts of the United States, are for the most part not in a position to obtain any decent evidence about the consequences of the uses that are made of their tests. What responsibilities and actions publishers can reasonably be expected to…

A summary of the writing assessment programs reviewed in this journal is presented. The problems inherent in the programs are outlined. A coordinated research program on major problems in writing assessment is proposed as being beneficial and cost-effective. (DWH)

The relationships between ratings on the Idaho Alternate Assessment (IAA) for 116 students with significant disabilities and corresponding ratings for the same students on two norm-referenced teacher rating scales were examined to gain evidence about the validity of resulting IAA scores. To contextualize these findings, another group of 54…

This paper was presented with other papers in a forum dealing with statewide testing programs. The primary purpose of the paper is to address practical considerations and methods of resolution for large districts or states who are planning on conducting large scale testing or assessment programs with criterion or performance referenced measures.…

The Arizona Tax Research Association analyzed the Arizona Teacher Proficiency Examination (ATPE). An item analysis of the 150 questions administered to 2,430 applicants from July 1982 through March 1983 was undertaken. A minimum point biserial correlation coefficient of .33 indicated the validity of an item. Nearly two-thirds (64 percent) of the…