Researchers in many disciplines work with ranking data. This data type is unique in that it is often deterministic in nature (the ranks of items "k"-1 determine the rank of item "k"), and the difference in a pair of rank scores separated by "k" units is equivalent regardless of the actual values of the two ranks in…

The importance of engaging in high quality program evaluation is a generally accepted principle underscored by external pressure from funders. High quality evaluation necessarily begins with good evaluation planning. This paper outlines Evolutionary Evaluation and specifically the Systems Evaluation Protocol, an approach that emphasizes…

Sexual assault on college campuses has become an epidemic requiring institutions to respond. Through Title IX, all educational institutions receiving federal financial assistance from the U.S. Department of Education must have a Title IX coordinator and must respond to allegations of sexual assault. Little specific guidance for how to…

In this chapter, two student members of the Student Voice Collaborative (SVC) describe their experiences as "Student Shadows" during the annual Quality Review process, used throughout the New York Department of Education to evaluate how well schools are organized to support student achievement. They chronicle how this experience enhanced…

Collaborative evaluation is an approach that offers, among others, many advantages in terms of access to information, quality of information gathered, opportunities for creative problem-solving, and receptivity to findings. In the last decade, collaborative evaluation has grown in popularity along with similar participatory, empowerment, and…

In many less developed democracies Voluntary Organizations for Professional Evaluation (VOPEs) face the challenges of low demand for evaluation and the resulting low economic capacity of national evaluation communities. The VOPE model that evolved in well-developed democracies is not directly applicable under these circumstances, so a new model…

Schools, districts, and state-level educational organizations are experiencing a great shift in the way they do the business of education. This shift focuses on accountability, specifically through the expectation of the effective utilization of evaluative-focused efforts to guide and support decisions about educational program implementation. In…

Composition assessment scholars have exhibited uneasiness with the language of norming grounded in distaste for the psychometric assumption that achievement of consensus in a communal assessment setting is desirable even at the cost of individual pedagogical values. Responding to the problems of a "reliability" defined by homogenous agreement,…

Collaborative Evaluation systematically invites and engages stakeholders in program evaluation planning and implementation. Unlike "distanced" evaluation approaches, which reject stakeholder participation as evaluation team members, Collaborative Evaluation assumes that active, on-going engagement between evaluators and program staff,…

Regressions that control for confounding factors are the workhorse of evaluation research. When treatment effects are heterogeneous, however, the workhorse regression leads to estimated treatment effects that lack behavioral interpretations even when the selection on observables assumption holds. Regressions that use propensity scores as weights…

The use of theory-driven evaluation is an emerging practice in the military--an aspect generally unknown in the civilian evaluation community. First developed during the 1991 Gulf War and applied in both the Balkans and Afghanistan, these techniques are now being examined in the North Atlantic Treaty Organisation (NATO) as a means to evaluate the…

Across the nation, states and districts are in the process of building better teacher evaluation systems that not only identify highly effective teachers but also systematically provide data and feedback that can be used to improve teacher practice. "A Practical Guide to Designing Comprehensive Teacher Evaluation Systems" is a tool…

Program evaluation and planning is at the heart of efforts to integrate the domains of practice and research. Traditionally, research and practice have operated in independent spheres with practitioners focused on the implementation of programs that affect individual behavior and researchers focused on the development and testing of theory.…

Autocorrelated residuals in longitudinal data are widely reported as common to longitudinal data. Yet few, if any, researchers modeling growth processes evaluate a priori whether their data have this feature. Sivo, Fan, and Witta (2005) found that not modeling autocorrelated residuals present in longitudinal data severely biases latent curve…

Teacher evaluations are often designed to serve two purposes: to measure teacher competence and to foster professional development and growth. A teacher evaluation system should give teachers useful feedback on classroom needs, the opportunity to learn new teaching techniques, and counsel from principals and other teachers on how to make changes…