Argumentation, a key scientific practice presented in the "Framework for K-12 Science Education," requires students to construct and critique arguments, but timely evaluation of arguments in large-scale classrooms is challenging. Recent work has shown the potential of automated scoring systems for open response assessments, leveraging…

Automated scoring of free drawings or images as responses has yet to be used in large-scale assessments of student achievement. In this study, we propose artificial neural networks to classify these types of graphical responses from a TIMSS 2019 item. We are comparing classification accuracy of convolutional and feed-forward approaches. Our…

Purpose: The purpose of this paper is to compare two years of results for one state's performance-based assessments for principal licensure Performance Assessment for Leaders (PAL). This includes the field trial (2014-2015) and first year of statewide implementation (2015-2016) when passing score requirements and fees were added. Survey results on…

The current study, a preliminary examination of whether music teachers are more susceptible to vocal problems than teachers of other subjects, asked for expert evaluation of audio recordings from licensed speech-language pathologists. Participants (N = 41) taught music (n = 23) or another subject (n = 18) in either elementary (n = 21), middle (n =…

This study examines the practical implications of using state tests to measure student achievement in impact evaluations that span multiple states and grades. In particular, the study examines the sensitivity of impact findings to (1) the type of assessment used to measured achievement (state tests or an external assessment administered by the…

Describes a method for coding themes directly from audiotapes without transcription and issues and methods by which reliability can be established. A study of concerns of professionals being trained on 11 school-based support teams coded by 2 raters and based on 19 hours of audiotaped interviews illustrates the method. (SLD)

Evaluation utilization/use is discussed with the concept of user rationality (including examples where the rational use of evaluation results may be unclear or may be nonuse). A current project focuses on studying the process of evaluation information use and on validating a theory of use. (Author/LC)

Based on similar studies conducted in 1978 and 1983, this paper provides information on changes in state department of education (SEA) evaluation units over a 5-year period and an update on current conditions. In brief, there appears to have been substantial reductions in the size of SEA evaluation unit staffs. While the majority of studies…

Perceptions of the current and ideal functioning roles of the principal were compared using responses of 147 Alabama teachers. Differences between current and ideal roles were found for categories of counselor, evaluator, motivator, and supervisor. These results emphasize the varying perceptions teachers have of principals. (SLD)

This paper reports on a study of the foundation and application of two procedures used to specify appropriate weights to be applied to components in determining the overall quality of a school. These procedures are multiattribute utility technology (MAUT) and policy capturing, and the paper presents the results of applying them, using key…

Reviews of 2,554 educational software programs by thirty reviewing sources were examined to compare the tendency of different review sources to give generally positive, negative, or neutral ratings. The reliability of language arts and reading software reviews was studied. Thirty-eight percent of the software programs were reviewed by two or more…

Evaluation procedures used in teacher performance assessment systems in 12 states of the Southern Regional Education Board area are described. Twenty-two trained evaluators from 10 states viewed videotaped lessons and applied their own evaluation standards to enable comparison of outcomes of evaluation. Similarities and differences are discussed.…

The New Standards Project is a joint effort of the Learning Research and Development Center (LRDC) at the University of Pittsburgh (Pennsylvania) and the National Center on Education and the Economy toward creation of a national examination system based on performance assessments. This study explored the feasibility of comparing performance on…

Methods that might be used to establish standards of performance that will permit the National Board for Professional Teaching Standards to classify candidate teachers as highly accomplished (worthy of National Board certification) or less than highly accomplished (not worthy of certification) are contrasted. Plans for research on these issues are…

Ten states participated in a cross-state scoring workshop in 1991, evaluating writing from elementary school, middle school, and high school students. Correlation of scores assigned by readers from one state with those from readers from another state were generally quite high. Implications for defining common standards are discussed. (SLD)