The long-term goal of the State Systemic Improvement Plan (SSIP) and other federal and state early intervention and early childhood education initiatives is improved child and family outcomes. States play a critical role in supporting practitioners in the use of evidence-based practices to improve child and family outcomes. When practitioners…

The purpose of this study is to further enhance the existing reliability and validity evidence related to the School Implementation Scale (SIS) and its use for evaluating the implementation of different integrated academic and behavior tiered Response-To-Intervention (RTI) models. Previous methodology conducted for validation of the instrument has…

Community-based education programs must demonstrate effectiveness to various funding sources. The pilot study reported here (funded by CYFAR, NIFA, USDA award #2008-41520-04810) had the goal of determining if state level programs with varied curriculum could use a common evaluation tool to demonstrate efficacy. Results in parenting and youth…

This article proposes that sampling design effects have potentially huge unrecognized impacts on the results reported by large-scale district and state assessments in the United States. When design effects are unrecognized and unaccounted for they lead to underestimating the sampling error in item and test statistics. Underestimating the sampling…

Systematic evaluation is an essential tool for understanding program effectiveness. This article describes the pilot test of a statewide evaluation tool for the Supplemental Nutrition Assistance Program-Education (SNAP-Ed). A computer algorithm helped Community Nutrition Educators (CNEs) build surveys specific to their varied educational settings…

This study is an evaluation of the psychometric issues associated with estimating objective level scores, often referred to as "subscores." The article begins by introducing the concepts of reliability and validity for subscores from statewide achievement tests. These issues are discussed with reference to popular scaling techniques, classical…

Using data from state and academic reports, an independent committee of researchers has evaluated the Massachusetts Teacher Tests. Scores are found to be highly unreliable, and the tests are found to contain questionable content. Suspending use of the tests is recommended. (SLD)

The critique of the Massachusetts Teacher Tests by W. Haney and others points out some flaws in the tests but ignores the fact that the tests provide some useful information to guide teacher selection decisions. Calls for additional study of these teacher evaluation instruments. (SLD)

Three new indicators of psychometric quality for objectives-based statewide assessments are proposed. These measures provide indication of the stability of reported data on item and objectives mastery, the validity of assessment items for members of various cultural groups, and the convergent validity of prescribed objectives mastery scores. The…

The Pennsylvania Plan for Educational Quality Assessment is discussed as to its beginnings, its goals, its implementation, and its future. The plan aids schoolmen of the State in assessing the quality of education their schools are providing in relation to 10 goals, which propose to encourage the students to develop self-understanding,…

In response to the need to develop reliable methods of evaluating Texas students' achievement of essential competencies, this study was designed to develop multiple-choice test items in food and nutrition in all comprehensive homemaking education courses, the semester course in food and nutrition, and all occupational methods of teaching food and…

The Regents Competency Program has two basic purposes: (1) to assure the early identification of students who need help in developing reading, writing, and mathematics skills and (2) to assure that students have acquired adequate competency in these skills before receiving a high school diploma. The New York State Preliminary Competency Test in…

A project provided teachers and other school personnel with a set of practical instruments for assessing students' generalizable mathematics skills in the secondary vocational training programs in Illinois. The major accomplishments of the project were advisory committee identification and selection; population and sample identification and…

Key findings of a pilot study of the Alaska Instructional Diagnostic System (AIDS) are summarized. The AIDS pilot test served to verify the appropriateness of the skills survey as well as the validity and reliability of the items. The AIDS testing system includes three components: (1) upper level skills surveys (grades 3-8); (2) lower level skill…

The statewide teaching performance assessment instruments being used in Georgia, North Carolina, and Florida were examined. Forty-one reliability and validity studies regarding the instruments in use in each state were collected from state departments and universities. Georgia uses the Georgia Teacher Performance Assessment Instrument. North…