Generalizability theory was used to examine the generalizability and dependability of outcomes from two single-item Direct Behavior Rating (DBR) scales: DBR of actively manipulating and DBR of visually distracted. DBR is a behavioral assessment tool with specific instrumentation and procedures that can be used by a variety of service delivery…

During the last 20 years, youth programming has shifted from risk reduction to youth development. While numerous instruments exist to measure selected individual characteristics/competencies among youth, a comprehensive instrument to measure four constructs of personal and social skills could not be identified. The purpose of this study was to…

As Americans shift their work and leisure activities online, Extension seeks to remain viable by delivering programs through a website known as eXtension. eXtension is predicated on the voluntary labor of Extension specialists and educators who form Communities of Practice to create and deliver content through the website. Evaluation of eXtension…

Scores assigned by trained raters and by an automated scoring system (SpeechRater[TM]) on the speaking section of the TOEFL iBT[TM] were validated against a communicative competence criterion. Specifically, a sample of 555 undergraduate students listened to speech samples from 184 examinees who took the Test of English as a Foreign Language…

Background: Children with high-functioning autism spectrum disorders (ASD) are at high risk for developing significant anxiety. Anxiety can adversely impact functioning across school, home and community environments. Cognitive behavioral therapies (CBT) are frequently used with success for children with anxiety symptoms. Modified CBT interventions…

By the end of their studies, non-native speakers of English studying at English-medium universities have had several years of exposure to English in that setting. Do non-native students, particularly those enrolled in non-language related programs, show different levels of second language (L2) speaking ability in their final semester of studies…

In this research, we investigated the suitability of implementing "e-rater"® automated essay scoring in a high-stakes large-scale English language testing program. We examined the effectiveness of generic scoring and 2 variants of prompt-based scoring approaches. Effectiveness was evaluated on a number of dimensions, including agreement…

In the past few years researchers have published lists of evidence-based practices that research indicates "should" be in place in programs and supports for students with emotional or behavioral disorders (EBD) (e.g., Farley, Torres, Wailehua, & Cook, 2012; Ryan, Pierce, & Mooney, 2008; Simpson, Peterson, & Smith, 2011). Some…

The objective of this research is to detect effective factors in English pronunciation teaching from the viewpoint of EIL (English as an International Language). This research is based on the data of the Successful Pronunciation Learners (SPL) selected by the raters, Japanese and native speakers of English (NSE), in Tominaga (2005). On the grounds…

The purpose of this study was to examine the ability of a Child Protective Services (CPS) screening and investigation process to identify children with developmental disabilities. The study used an emergent design, ethnographic interviews, purposive sampling, inductive data analysis, and grounded theory building. Ethnographic interviews were…

Internal evaluators encounter risks but also significant opportunities to strengthen organizational and professional ethics. Potential contributions depend, in part, on the conjunction of ethics and evaluation in the role of the internal evaluator as the person specially commissioned to investigate value and render judgment based on evidence…

Background: A music education program with a goal of enhancing cognitive development of preschool-aged children enrolled in local preschools is evaluated by The Evaluation Center at Western Michigan University. The budget for the evaluation was small, and therefore presented several challenges to the evaluation team. Purpose: Through a case study…

Although many leaders in evaluation advise evaluators to address the diverse needs of stakeholders, very little is known about how or if practicing evaluators address this injunction. Understanding how practicing evaluators address the needs of multiple stakeholders could inform evaluator training. The purpose of this qualitative study was to…

What kinds of ethical dilemmas do people who are new to the profession of evaluation face? Are they different from the challenges faced by more experienced evaluators? What strategies do they use to resolve them? These questions were the starting point for the essays included in this issue of the Ethical Challenges section. The essays were written…

Programs and policies invariably contain new knowledge. Theories about knowledge utilization, diffusion, implementation, transfer, and knowledge translation theories illuminate some mechanisms of change processes. But more often than not, when it comes to understanding patterns about change processes, "the foreground" is privileged more…