Onscreen marking (OSM) has been used for the majority of Hong Kong public examinations since 2012. The current study compares marker reactions to OSM, ie, perceived ease of use and acceptance of OSM, against the backdrop of virtually all subject areas being marked on screen. The data were collected from three major sources: (1) survey data…

This study examined contributions of lexical factors to native-speaking raters' assessments of comprehensibility (ease of understanding) of second language (L2) speech. Extemporaneous oral narratives elicited from 40 French speakers of L2 English were transcribed and evaluated for comprehensibility by 10 raters. Subsequently, the samples were…

Exercise physiology courses have transitioned to competency based, forcing Universities to rethink assessment to ensure students are competent to practice. This study built on earlier research to explore rater cognition, capturing factors that contribute to assessor decision making about students' competency. The aims were to determine the source…

This study investigates the relative impact of verbal expression and tone of voice when native speakers of English form an impression of non-native speech. Four expressions of inquiry uttered in two tones by non-native speakers were judged by native listeners and analyzed using an ordinal Probit model. Plain expressions received lower scores than…

Evaluating initiatives implemented across multiple settings can elucidate how various contextual factors may influence both implementation and outcomes. Understanding context is especially critical when the same program has varying levels of success across settings. We present a framework for evaluating contextual factors affecting an initiative…

Service-learning courses provide students with practical opportunities to enhance their learning and development in the field, along with getting students engaged in different communities and settings. However, there are still many challenges to designing and offering effective service-learning courses, such as requiring all students to…

Knowledge brokering is a promising practice for addressing the challenge of using research evidence, including evaluation findings, in policy implementation. For public policy practitioners, it means playing the role of an intermediary who steers the flow of knowledge between producers (researchers) and users (decision makers). It requires a set…

Drawing on a qualitative study of an educational reform and its external evaluation, I describe how a well-intentioned but poorly conceptualized evaluation helped perpetuate asymmetries in the generation and use of evaluation findings. This article explores this project's failure to clarify evaluator roles, identify intended users and expected…

Problem-based learning has been applied over the last three decades to a diverse range of learning environments. In this educational approach, different problems are posed to the learners so that they can develop different solutions while learning about the problem domain. When applied to conceptual modelling, and particularly to Qualitative…

Novice members of a Norwegian national rater panel tasked with assessing Year 8 pupils' written texts were studied during three successive preparation sessions (2011-2012). The purpose was to investigate how the raters successfully make use of different decision-making strategies in an assessment situation where pre-set criteria and standards give…

This study researched the effective practices that should be utilized when evaluating faculty. In addition to how faculty are evaluated, the study looked at how faculty evaluations are connected to faculty development for improvement opportunities. This study focused on the 15 Iowa community colleges. Each community college was asked to provide…

A short training program for evaluating responses to an essay writing task consisted of scoring 20 training essays with immediate feedback about the correct score. The same scoring session also served as a certification test for trainees. Participants with little or no previous rating experience completed this session and 14 trainees who passed an…

We addressed Deville and Chalhoub-Deville's (2006), Schoonen's (2012), and Xi and Mollaun's (2006) call for research into the contextual features that are considered related to person-by-task interactions in the framework of generalizability theory in two ways. First, we quantitatively synthesized the generalizability studies to determine the…

The purpose of this research was to develop a measure of sport injury anxiety (SIA), defined as the tendency to make threat appraisals in sport situations where injury is seen as possible and/or likely. The Sport Injury Anxiety Scale (SIAS) was developed in three stages. In Stage 1, expert raters evaluated items to determine their adequacy. In…

This article presents a discourse analytic study of how the concept of impartiality is socially constructed by members of the development aid community through an examination of linguistic traits and patterns within (a) inter- and intraorganizational interactions and (b) relevant aid evaluation policy documents. A qualitative analysis of…