In this study, the authors examined the relations among clinician ratings of treatment improvement and discrepancies between parent and blinded laboratory rater reports of child social functioning administered before and after treatment for social anxiety disorder. Participants included a clinic sample of 101 children (7-16 years old; M = 11.67,…

Blending high-quality and rigorous research with pure evaluation practice can often be best accomplished through thoughtful collaboration. The evaluation of a high school drug prevention program (All Stars Senior) is an example of how perceived competing purposes and methodologies can coexist to investigate formative and summative outcome…

Although children with social phobia (SP) and selective mutism (SM) present similarly in a clinical setting, it remains unclear whether children with SM are unable to speak due to overwhelming anxiety, or whether withholding speech functions as an avoidance mechanism. A total of 35 children (ages 5-12 years) with either SM (n = 10), SP (n = 11),…

Much has been written about the evaluation of faculty research productivity in promotion and tenure decisions, including many articles that seek to determine the rank of various marketing journals. Yet how faculty evaluators combine journal quality, quantity, and author contribution to form judgments of a scholar's performance is unclear. A…

The continuously growing demand for program results has produced an increased need for evaluation capacity building (ECB). The "Integrative ECB Model" was developed to integrate concepts from existing ECB theory literature and to structure a synthesis of the empirical ECB literature. The study used a broad-based research synthesis method with…

Making the program theory explicit is an essential first step in Theory Driven Evaluation (TDE). Once explicit, the program logic can be established making necessary links between the program theory, activities, and outcomes. Despite its importance evaluators often encounter situations where the program theory is not explicitly stated. Under such…

Legitimate knowledge claims about causation have been a central concern among evaluators and applied researchers for several decades and often have been the subject of heated debates. In recent years these debates have resurfaced with a renewed intensity, due in part to the priority currently being given to randomized experiments by many funders…

Many studies have shed light on rater criteria for assessing the performance of language skills (e.g., Eckes, 2005). However, the interface between rater assessment and interlanguage pragmatics (ILP) has remained largely unnoticed. To address this interface, this study explored the ratings native (NES) and nonnative English speaking (NNES)…

Quality and accountability mandates require institutions to monitor online instruction in a uniform and complete manner. In many institutions, instructor training is sparse and faculty evaluation occurs only through end-of-course student evaluations that may or may not yield adequate information on how the instructor performs online. Consequently,…

Background: The effectiveness of cognitive-behavioral treatment (CBT) in inner city schools, when delivered by novice CBT clinicians, and compared to usual care (UC), is unknown. Objective: This pilot study addressed this issue by comparing a modular CBT for anxiety disorders to UC in a sample of 32 volunteer youth (mean age 10.28 years, 63%…

Experience indicates that the questioning and democratic nature of the community of enquiry can be demanding and unsettling for teachers, presenting unaccustomed challenges and moral dilemmas. This paper argues that such significant episodes in the practice of Philosophical with Children (PwC) offer rich opportunities for wider critical reflection…

A sample of educational evaluators was asked to design an outcomes-focused evaluation of a school program that aims to improve the academic achievement and self-esteem of students. Evaluators provided detailed descriptions of their evaluation design and methodology in their responses. These descriptions were coded and analyzed to determine the…

Questioning and discussion techniques are effective instructional methods that develop critical thinking and problem solving skills. Despite these positive associations, there is inconsistent implementation of these techniques across disciplines and grade levels, often resulting in a negative effect on student achievement. This collective case…

There is a growing recognition of the value of measuring fidelity of implementation (FOI) as a necessary part of evaluating interventions. However, evaluators do not have a shared conceptual understanding of what FOI is and how to measure it. Thus, the creation of FOI measures is typically a secondary focus and based on specific contexts and…

Practitioners may be called upon to assess adults who have alleged child abuse as a minor and are seeking reparations. Such assessments may be used by the courts to determine harm and assess damages related to their claim or testimony. Our clinical/research team has conducted many such evaluations and reported the findings pertaining to the…