Practical constraints in rater-mediated assessments limit the availability of complete data. Instead, most scoring procedures include one or two ratings for each performance, with overlapping performances across raters or linking sets of multiple-choice items to facilitate model estimation. These incomplete scoring designs present challenges for…

Increasingly surveys are using interviewers to collect objective health measures, also known as biomeasures, to replace or supplement traditional self-reported health measures. However, the extent to which interviewers affect the (im)precision of biomeasurements is largely unknown. This article investigates interviewer effects on several…

Rater effects, or raters' tendencies to assign ratings to performances that are different from the ratings that the performances warranted, are well documented in rater-mediated assessments across a variety of disciplines. In many real-data studies of rater effects, researchers have reported that raters exhibit more than one effect, such as a…

Researchers have explored a variety of topics related to identifying and distinguishing among specific types of rater effects, as well as the implications of different types of incomplete data collection designs for rater-mediated assessments. In this study, we used simulated data to examine the sensitivity of latent trait model indicators of…

Evaluators can work with brief units of text-based data, such as open-ended survey responses, text messages, and social media postings. Online crowdsourcing is a promising method for quantifying large amounts of text-based data by engaging hundreds of people to categorize the data. To further develop and test this method, individuals were…

An antecedent exercise (AE) intervention was conducted with two young children who demonstrated challenging behaviors during a large group circle time activity in a preschool classroom; outcomes were evaluated in the context of alternating treatments designs. Results from the initial visual analysis suggested no overlap between conditions, with…

Onscreen marking (OSM) has been used for the majority of Hong Kong public examinations since 2012. The current study compares marker reactions to OSM, ie, perceived ease of use and acceptance of OSM, against the backdrop of virtually all subject areas being marked on screen. The data were collected from three major sources: (1) survey data…

This article presents a content analysis of the 2013 School Leadership Program (SLP) grants. SLP projects provide a unique opportunity for participants in the field to explore innovative leadership preparation and development and their impact on program participants, schools, school districts, and students. The article begins with an overview of…

Randomized controlled trials are the gold standard in research but may not fully explain or predict outcome variations in community-based interventions. Demonstrating efficacy of externally driven programs in well-controlled environments may not translate to community-based implementation where resources and priorities vary. A bottom-up evaluation…

Program and/or course evaluation is a process in which different types of data are collected systematically in order to study the virtues and weaknesses of a language instruction program. Program evaluation is, in fact, one of the essential aspects of any curriculum. It is a kind of quality control in which various aspects of an instructional…

This report focuses on the progress made and challenges faced by the nine states in achieving the first three objectives for which they were held accountable for use of their Race to the Top Early Learning Challenge (RTTELC) funds. It discusses discrete findings related to these objectives in relevant chapters of the report: Tiered Quality Rating…

This investigation seeks to understand "action research" as an approach to "interactive form of evaluation". The first half of the investigation illuminates the approach with the help of the selective body of literature and the second half draws attention to its application in the field with the help of an authentic evaluation…

This study is a content analysis of selected federally funded leadership preparation program evaluation proposals for the 2008, 2009, and 2010 School Leadership Program grants. The United States Department of Education (USDE) began awarding funding in 2002 to develop and implement preparation programs for aspiring and current assistant principals…

In the absence of any national quality performance monitoring system (until 2006) 8 of 21 Greek public universities voluntarily participated in the European University Association Institutional Evaluation Programme (EUA-IEP). Empirical data were collected through documents (eight evaluators' reports). This article uses qualitative content analysis…

In 1999, the National Science Foundation (NSF) awarded funds to the Evaluation Center at Western Michigan University to conduct an external evaluation of the Advanced Technological Education (ATE) program. ATE, a federally mandated program designed to increase the number and quality of skilled technicians in the U.S. workforce, has funded over 346…