The purposes of this study were to investigate the effects of rater training in an L2 performance assessment and to examine the eligibility of L2 users of English as raters in L2 performance assessment. Rater training was conducted in order for raters to clearly understand the criteria, the evaluation items, and the evaluation procedure. In this…

This article addresses the question, "What factors contributed to successful completion of a needs and resource assessment in local communities, using the empowerment evaluation approach?" Case studies of three successful projects allow the exploration of organizational, community, state, and national factors that contributed to a strong needs and…

The purpose of this study was to highlight the role of twin designs in understanding children's conversational interactions. Specifically, we (a) attempted to replicate the findings of genetic effects on children's conversational language use reported in DeThorne et al. (2008), and (b) examined whether the language used by examiners in their…

This article details an investigation into onscreen marking (OSM) in Hong Kong--where paper-based marking (PBM) is being phased out, to be completely superseded by OSM. It is a specific follow-up to a larger study (Coniam, 2009a) involving 30 raters who had previously rated English language essay scripts on screen in the 2007 Hong Kong Certificate…

This study investigates the effects of band labels on evaluators' judgments of musical performance. High school concert band members (n = 72), wind ensemble members ( n = 77), and band directors (n = 8) were randomly assigned to a band label or no label group. Only the band label group was given evaluation forms that specified the group playing…

For those seeking ways by which they might serve more knowledgeably and effectively as members of their institution's leadership team--particularly when it comes to issues of institutional assessment and effectiveness--the author strongly recommends active participation in any aspect of the accreditation/reaffirmation process, including serving as…

Evaluation researchers and practitioners acknowledge that involving stakeholders in the planning and implementation of an evaluation increases buy-in, understanding, and use. With the recent increase in multi-site evaluations of large federal programs, evaluators must think differently about how to encourage meaningful collaboration by…

How do evaluation practitioners make choices when they evaluate a program? What function do evaluation theories play in practice? In this article, I report on an exploratory study that examined evaluation practices in France. The research began with observations of practitioners' activities, with a particular focus on the phases of evaluation…

The competencies required for conducting assessment in the educational context have recently been reformulated in view of social constructivist perspectives and the acknowledgement of the role of classroom assessment in promoting learning. These changes have impacted the knowledge base language assessors need to obtain, and hence the contents of…

Autocorrelated residuals in longitudinal data are widely reported as common to longitudinal data. Yet few, if any, researchers modeling growth processes evaluate a priori whether their data have this feature. Sivo, Fan, and Witta (2005) found that not modeling autocorrelated residuals present in longitudinal data severely biases latent curve…

This excerpt from the opening plenary asks evaluators to consider two questions regarding learning and evaluation: (a) How do evaluators know if, how, when, and what people are learning during an evaluation? and (b) In what ways can evaluation be a learning experience? To answer the first question, evaluators can apply the commonplaces of…

The Accrediting Council for Independent Colleges and Schools ("ACICS" or "Council") supports the concept that periodic evaluation entered into voluntarily by institutions and their peers enhances the quality of the educational process and demonstrates that self regulation of a profession is superior to outside regulation. This has been the…

Background: Cultural competency is an important but under-adopted skill among professional evaluators. Yet in the transactions around job seeking and hiring in evaluation, cultural competency is a practical and common concept. How cultural competency gets communicated in those transactions may provide insights for the field. Purpose: The purpose…

Teacher evaluations are often designed to serve two purposes: to measure teacher competence and to foster professional development and growth. A teacher evaluation system should give teachers useful feedback on classroom needs, the opportunity to learn new teaching techniques, and counsel from principals and other teachers on how to make changes…

The study reported here examined the evaluation criteria used to assess the proficiency and effectiveness of the language produced in an oral performance test of English conducted in an American university context. Empirical methods were used to analyze qualitatively and quantitatively transcriptions of the Oral English Tests (OET) of 44…