More evaluators have anchored their work in equity-focused, culturally responsive, and social justice ideals. Although we have a sense of approaches that guide evaluators as to how they should attend to culture, diversity, equity, and inclusion (DEI), we have not yet established an empirical understanding of how evaluators measure DEI. In this…

The factors that influence rater scoring have been a subject of great interest to researchers in second language assessment. However, the research on the impact of test-takers' speech profiles (e.g., a jagged or a flat profile reflecting analytic subscores) on raters' scoring behaviors remains to be seen. To investigate the role of speech profiles…

"Measure, Use, Improve! Data Use in Out-of-School Time" shares the experience and wisdom from a broad cross-section of out-of-school time professionals, ranging from internal evaluators, to funders, to researchers, to policy advocates. Key themes of the volume include building support for learning and evaluation within out-of-school time…

Second-language (L2) testing researchers have explored the relationship between speakers' overall speaking ability, reflected by holistic scores, and the speakers' performance on speaking subcomponents, reflected by analytic scores (e.g., McNamara, 1990; Sato, 2011). These research studies have advanced applied linguists' understanding of how…

Despite growing interest in data visualization and graphically aided reporting, the evaluation literature could benefit from additional guidance on systematically integrating visual communication design and marketing into comprehensive communication strategies to improve data dissemination. This article describes the role of targeted communication…

In Michigan, 98 percent of teachers were rated effective or better under new teacher-evaluation systems recently put in place. In Florida, 97 percent of teachers were deemed effective or better. Principals in Tennessee judged 98 percent of teachers to be "at expectations" or better last school year, while evaluators in Georgia gave good…

It is a truism of American social policy that our nation has great success generating innovative programs that improve outcomes for participants--but that we are far less effective at moving from small, "boutique" programs into broadly applied solutions that improve the prospects of large numbers of individuals. This is certainly true in…

In 1999, the National Science Foundation (NSF) awarded funds to the Evaluation Center at Western Michigan University to conduct an external evaluation of the Advanced Technological Education (ATE) program. ATE, a federally mandated program designed to increase the number and quality of skilled technicians in the U.S. workforce, has funded over 346…

Background: A music education program with a goal of enhancing cognitive development of preschool-aged children enrolled in local preschools is evaluated by The Evaluation Center at Western Michigan University. The budget for the evaluation was small, and therefore presented several challenges to the evaluation team. Purpose: Through a case study…

Background: Cultural competency is an important but under-adopted skill among professional evaluators. Yet in the transactions around job seeking and hiring in evaluation, cultural competency is a practical and common concept. How cultural competency gets communicated in those transactions may provide insights for the field. Purpose: The purpose…

The purpose of this Guide is to advise researchers, policymakers, and others on when it is possible to conduct a high-quality randomized controlled trial in education at reduced cost. Well-designed randomized controlled trials are recognized as the gold standard for evaluating the effectiveness of an intervention (i.e., program or practice) in…

A stakeholder-focused approach to the assessment of racial/ethnic bias and gender bias in the Michigan State Court system is presented. The approach ensures that findings are used to improve the court's response to attorneys and litigants. The process should help evaluators work with client groups involved in politically charged issues. (TJH)

The primary purpose of this study was to investigate the impact of the level of observed performance on the feedback strategy selected by a performance evaluator. One hundred and twenty-three actual performance evaluators from 15 different organizations and 123 college students reviewed, in groups which ranged from 2 to 20, a job description for…

During workshops held in March and April of 2005, Michigan educators got a lesson in how to benchmark their work against promising practices in higher-performing schools with similar socioeconomic profiles. Teams of teachers, educational supervisors, and principals from 13 elementary and middle schools that had been struggling to make adequate…

This paper delineates the organizational structure and operational procedures of the Michigan Institute for Educational Management (MIEM) Assessment Center Program for prospective school principals and assistant principals. The program is part of the Assessment Center Project of the National Association of Secondary School Principals (NASSP). The…