This article reports findings from a research program exploring the role of mediation in an "adaptive learning" process through study of developmental evaluation (DE). Our study focuses on how mediators might influence the relationships between components of a social learning system and the implications for adaptive learning.…

The present research paper introduces a translation evaluation method called Calibrated Parsing Items Evaluation (CPIE hereafter). This evaluation method maximizes translators' performance through identifying the parsing items with an optimal p-docimology and d-index (item discrimination). This method checks all the possible parses (annotations)…

The rating of proficiency tests that use the Inter-agency Roundtable (ILR) and American Council on the Teaching of Foreign Languages (ACTFL) guidelines claims that each major level is based on hierarchal linguistic functions that require mastery of multidimensional traits in such a way that each level subsumes the levels beneath it. These…

The use of machine translation (MT) in the academic context has increased in recent years. Hence, language teachers have found it difficult to ignore MT, which has led to some concerns. Among the concerns, its accuracy has become a major factor that shapes language teachers' pedagogical decision to use MT in their language classrooms. Despite the…

Drawing on Glenn Fulcher's extensive work in performance-based language assessment of speaking, this paper explores the assessment of L2 speaking ability in local language testing contexts. For that purpose, I review Fulcher's influential work that highlights the relationship between the speaking construct, the task, the performance, and the…

Program reviews are standard practice in higher education. Yet, due to the infancy of the leadership discipline, little is known about the process of conducting reviews of leadership programs. Through interviewing 13 experienced leadership program reviewers in both curricular and co-curricular contexts, the authors of this study aim to address…

This study examined how L2 English speakers interpreted the notion of native English speakers (NESs) and nonnative English speakers (NNESs) and whether nativeness would influence their self-perception and speech production. It aimed at filling the following research gaps. First, limited studies have explored how L2 English speakers view the other…

George Grob presented the fifth and final Eleanor Chelimsky Forum address at the 2017 annual meeting of the Eastern Evaluation Research Society. In this commentary, I respond to several points that George raises in the "American Journal of Evaluation" paper based on that address. An overarching theme of my comments involves the potential…

Raters must score accurately and consistently for classroom observation scores to be valid. This requires (a) a standard defining when scoring is accurate and consistent enough and (b) measuring and remediating rater performance against that standard. Current practice has focused on this second problem to the exclusion of the first. My goal here…

This qualitative study was conducted to understand the use and impact of the Texas Principal Evaluation and Support System (T-PESS) as a leadership framework and determine if patterns in leadership activities and behaviors could be observed and identified. Observations and interviews were conducted with four principal evaluators and four…

Pressure on evaluators has been investigated recently by surveys in the USA, the UK, Germany, and Switzerland. This study compares the results of those studies regarding pressure on evaluators in different countries. The findings suggest that independence of evaluations does not exist for many respondents. Moreover, the person who commissioned the…

The current study sought to examine whether explicit instruction affects teacher evaluators in terms of a six-factor Teacher Evaluation (TE) model encompassing Perception, Method, System, Content, Purpose, and Outcome. Moreover, it explored how training of the evaluators affects their practices in actual evaluation. To this end, twenty EFL…

"Contract cheating," instances in which a student enlists someone other than themselves to produce coursework, has been identified as a growing problem within academic integrity literature and in news headlines. The percentage of students who have used this type of cheating has been reported to range between 6% and 15.7%. Generational…

This study examines the putative benefits of explicit phonetic instruction, high variability phonetic training, and their effects on adult nonnative speakers' Mandarin tone productions. Monolingual first language (L1) English speakers (n = 80), intermediate second language (L2) Mandarin learners (n = 40), and L1 Mandarin speakers (n = 40) took…

In an era of widespread automation--from grocery store self-checkout machines to selfdriving cars--it is not outrageous to wonder: can teachers be automated? And more specifically, can automated computer teachers instruct students how to write? Automated computer programs have long been used in summative writing evaluation efforts, such as scoring…