This paper provides a reflective account of a consultation process on professional designations for evaluators initiated and coordinated by the Canadian Evaluation Society (CES). Described are: (1) the forces leading CES to generate discussion and debate about professional designations for Canadian evaluators, (2) the process of developing and…

Depth, resonance, and shadow are added to how the expertise of managing evaluation could be grasped and understood. Returning to the notion of "competency" (Stevahn, King, Ghere, & Minnema, 2005), this is contrasted more deeply with Dreyfus and Dreyfus's (2004) conception of expert and to its use in nursing by Benner (2004; Benner, Tanner, &…

Assessment personnel are those individuals who work in the capacity of evaluation of students with disabilities, including, but not limited to, educational diagnosticians, educational examiners, psychometrists, and instructional specialists. These professionals are responsible for identifying strengths and weaknesses and for providing teachers…

No self-report measure of cultural competence currently exists in program evaluation. Adapting items from cultural competence measures in fields such as counseling and nursing, the researchers developed the Cultural Competence of Program Evaluators (CCPE) self-report scale. The goals of this study were to validate the CCPE and to assess…

This study investigated whether raters' second language (L2) background and the first language (L1) of test takers taking the TOEFL iBT® Speaking test were related through scoring. After an initial 4-hour training period, a group of 107 raters (mostly of learners of Chinese, Korean, and Spanish), listened to a selection of 432 speech samples that…

This manuscript shares lessons learned from conducting an action evaluation of the use of multimedia case studies in Management Information Systems (MIS) courses. Three undergraduate MIS classes took part in the study. The purpose for using case studies in these classes was to teach students about the role of MIS in business. An action evaluation…

Evaluators are frequently asked to assess the effectiveness of school programs implemented to improve academic achievement. School connectedness has been shown to be directly related to academic achievement (McNeely, Nonnemaker, & Blum, 2002) and is therefore of interest to evaluators. The construct of school connectedness has been shown to…

We examined whether instructional materials describing how to rate child ADHD symptoms would improve the accuracy of mothers' ratings of ADHD symptoms presented in standard child behavior stimuli, and whether instructions would be equally effective across a range of maternal depressive symptoms and family incomes. A community sample of 100 mothers…

This paper reports the findings of an empirical study on ESL/EFL teachers' evaluation and interpretation of oral English proficiency as elicited by the national College English Test-Spoken English Test (CET-SET) of China. Informed by debates on the issue of native speaker (NS) norms which have become the focus of attention in recent years, this…

Background: Since the late 1970s participatory approaches have been widely promoted to evaluate international development programs. However, there is no universal agreement of what is meant by participatory evaluation. For some evaluators, participatory evaluations involve the extensive participation of all stakeholder groups (from donor to…

The interrater reliability of the Supports Intensity Scale (SIS) was investigated under the condition that interviewers had to have been trained and/or experienced in its administration and scoring. Both corrected and noncorrected Pearson's product-moment coefficients were generated to assess interinterviewer, interrespondent, and mixed interrater…

Evaluators know they are supposed to identify and engage with stakeholders. What happens when the client has a very narrow concept of the meaning of evaluation? What happens when the primary stakeholders, including the client, disappear just as the evaluation gets started? First, it is important to acknowledge the challenge, then develop a…

Each year a great many evaluations are conducted of international development efforts around the world. These development evaluations study projects, programs, country-wide portfolios, policy reform efforts, and other topics of interest to funders, governments, program managers, and other involved stakeholders. Although some of these evaluations…

A central task of evaluation is to use systematic inquiry to judge the merit, worth, and significance of programs. As a field in education, evaluation began to mature during the 1960s, when policy makers began funding researchers to study the impact of Great Society programs. Since that time, evaluation and policy-making have been intertwined,…

Research and clinical practices often rely on an utterance unit for spoken language analysis. This paper calls attention to the problems encountered when identifying utterance boundaries in young children's spontaneous conversational speech. The results of a reliability study of utterance boundary assignment are described for 20 females with…