While evaluators have explored the implementation of culturally responsive evaluation (CRE), the failures of applying CRE are less often told. In this article, we use a reflective case narrative to explore our successes and failures in implementing our CRE approach, including an educative stance. We draw on a formative evaluation of a…

This study applied a Systemic Functional Linguistics (SFL) model to explore how 27 first-year university students in two different English proficiency groups improved their lexicogrammatical choices and metafunctions for writing analytical exposition essays during a 15-week course. To explore how "the teaching learning cycle" influences…

When facilitating large-scale instructional change, leaders face stakeholder tensions that arise from different institutional pressures. Over the past 4 years, we have created an innovative "live" case competition in a strategic management course as our college's signature undergraduate experiential learning opportunity. This case has…

Background: This study aims to develop a low-stakes assessment tool to establish a classroom English language benchmark that Japanese teachers of English can use for their own professional development purposes. To start with, we describe the differences between CLA (Classroom Language Assessment) in Hong Kong and the IDS (Integrative Diagnostic…

We develop a random forest classifier that helps assign academic credit for a student's class forum participation. The classification target are the four classes created by student rank quartiles. Course content experts provided ground truth by ranking a limited number of post pairs. We expand this labeled set via data augmentation. We compute the…

This study compares the language proficiency gains of two groups of students taking a business English course module in a bilingual university in Catalonia (Spain). Whereas one of these groups followed a 'translanguaging' or 'plurilingual' pedagogy, the other followed a strictly monolingual approach. Participants were 54 mostly Catalan/Spanish…

The accreditation process and the federal role in shaping it are clearly of critical importance for ensuring all students have access to quality postsecondary options. And yet, due to the complexity of the relationship among the federal government, accreditors, and institutions, and the opacity of the accreditation process itself, there is little…

This paper addresses the place of the use of international "experts" in assuring the quality of Teacher Education programmes within Higher Education, in relation to the global agenda of improving standards. The experiences of one such "expert" are discussed and recommendations made as to the type of person suited to the role.…

We present the results of a study that explored the emotions experienced by students during interaction with an educational game for math (Heroes of Math Island). Starting from emotion frameworks in affective computing and education, we considered a larger set of emotions than in related research. For emotion labeling, we started from a standard…

Purpose: This study examines the degree to which adolescents in Iceland are judged by native Icelandic speakers to have a foreign accent both in Icelandic and in English, two languages that are learned under different conditions, as the community and school language, and through school and incidental exposure. Method: Fifty-eight adolescents, 27…

Research on the evaluation of large-scale public-sector reforms is rare. This article sets out to fill that gap in the evaluation literature and argues that it is of vital importance since the impact of such reforms is considerable and they change the context in which evaluations of other and more delimited policy areas take place. In our…

The Angoff standard setting method relies on content experts to review exam items and make judgments about the performance of the minimally proficient examinee. Unfortunately, at times content experts may have gaps in their understanding of specific exam content. These gaps are particularly likely to occur when the content domain is broad and/or…

What is different about the conduct of evaluations in conflict zones compared to nonconflict zones--and how do these differences affect (if at all) the ethical calculations and behavior of evaluators? When are ethical issues too risky, or too uncertain, for evaluators to accept--or to continue--an evaluation? These are the core questions guiding…

Ratings given to the same item response may have a stronger correlation than those given to different item responses, especially when raters interact with one another before giving ratings. The rater bundle model was developed to account for such local dependence by forming multiple ratings given to an item response as a bundle and assigning…

Evaluation capacity building (ECB) has progressed as a concept since it was the conference theme of the American Evaluation Association in the year 2000. This commentary poses some questions about underexamined issues in ECB about organizations, evaluators, and funders.