Community-based education programs must demonstrate effectiveness to various funding sources. The pilot study reported here (funded by CYFAR, NIFA, USDA award #2008-41520-04810) had the goal of determining if state level programs with varied curriculum could use a common evaluation tool to demonstrate efficacy. Results in parenting and youth…

This article proposes that sampling design effects have potentially huge unrecognized impacts on the results reported by large-scale district and state assessments in the United States. When design effects are unrecognized and unaccounted for they lead to underestimating the sampling error in item and test statistics. Underestimating the sampling…

This study evaluated large-scale state writing assessments for the inclusion of motivational characteristics in the writing task and written prompt. We identified 6 motivational variables from the authentic activity literature: time allocation, audience specification, audience intimacy, definition of task, allowance for multiple perspectives, and…

State-level testing programs continue to grow, and the challenge of validation does not wane. Although more than a decade has passed since the 1999 Joint Standards for Educational and Psychological Testing set out a call for the organization of validity evidence into validity arguments, practical examples of such arguments are not readily…

Systematic evaluation is an essential tool for understanding program effectiveness. This article describes the pilot test of a statewide evaluation tool for the Supplemental Nutrition Assistance Program-Education (SNAP-Ed). A computer algorithm helped Community Nutrition Educators (CNEs) build surveys specific to their varied educational settings…

This study is an evaluation of the psychometric issues associated with estimating objective level scores, often referred to as "subscores." The article begins by introducing the concepts of reliability and validity for subscores from statewide achievement tests. These issues are discussed with reference to popular scaling techniques, classical…

As part of a Response to Intervention (RTI) model, curriculum-based measurement (CBM) is an evidence-based assessment system that uses screening and formative assessments to help guide instruction. The two primary purposes of this study are to examine the relations between the easyCBM benchmark measures and a statewide large-scale assessment and…

A summary of the writing assessment programs reviewed in this journal is presented. The problems inherent in the programs are outlined. A coordinated research program on major problems in writing assessment is proposed as being beneficial and cost-effective. (DWH)

The relationships between ratings on the Idaho Alternate Assessment (IAA) for 116 students with significant disabilities and corresponding ratings for the same students on two norm-referenced teacher rating scales were examined to gain evidence about the validity of resulting IAA scores. To contextualize these findings, another group of 54…

State directors of teacher certification and testing provided updated data (reports, press clippings, manuals, legislative hearings, and validity studies) in April 1987 describing their testing programs and the latest passing rates. Twenty-seven states have admissions testing programs, and 44 have certification testing programs. It is unclear…

A method to investigate consequential evidence of validity for a state assessment developed to change teacher instructional practices is presented. Survey responses from over 1,000 Kansas teachers were used to construct a path model that allowed effects of the state assessment to be studied at building and teacher levels. (SLD)

Using data from state and academic reports, an independent committee of researchers has evaluated the Massachusetts Teacher Tests. Scores are found to be highly unreliable, and the tests are found to contain questionable content. Suspending use of the tests is recommended. (SLD)

The critique of the Massachusetts Teacher Tests by W. Haney and others points out some flaws in the tests but ignores the fact that the tests provide some useful information to guide teacher selection decisions. Calls for additional study of these teacher evaluation instruments. (SLD)

Reports on an ongoing evaluation of the Vermont portfolio assessment program. Indicates that the positive news about the instructional effects of the assessment program are in contrast with the empirical findings about the quality of the data the program has yielded. (SLD)

This study analyzed the scores of the 585 state FFA (Future Farmers of America) degree applications in Ohio in 1982. The study identified sex, year in school, area of state, and program area of applicants; influence of the scoring areas of the application; and relationships among the scoring areas. (CT)