Background: Mid-semester evaluations are gaining traction as a means to gather evaluation data for formative purposes. However, it is not clear if course coordinators who conduct these evaluations are adequately equipped with evaluative knowledge and skills to guide them through their evaluative processes. Objectives: This study is a…

While significant attention has been given to test equating to ensure score comparability, limited research has explored equating methods for rater-mediated assessments, where human raters inherently introduce error. If not properly addressed, these errors can undermine score interchangeability and test validity. This study proposes an equating…

Human raters' assessment of interpreting is a complex process. Previous researchers have mainly relied on verbal reports to examine this process. To advance our understanding, we conducted an empirical study, collecting raters' eye-movement and retrospection data in a computerised interpreting assessment in which three groups of raters (n = 35)…

In this article, the authors share their reflections on how different experiences and paradigms have broadened their understanding of the work of assessment in higher education. As they collaborated to create a panel for the 2024 International Conference on Assessing Quality in Higher Education, they recognized that they, as assessment…

This study examined the effect of rater training on promoting inter-rater reliability in oral language assessment. It also investigated whether rater training and the consideration of the examinees' expectations by the examiners have any effect on test-takers' perceptions of being fairly evaluated. To this end, four raters scored 31 Iranian…

The literature on Automated Essay Scoring (AES) systems has provided useful validation frameworks for any assessment that includes AES scoring. Furthermore, evidence for the scoring fidelity of AES systems is accumulating. Yet questions remain when appraising the scoring performance of AES systems. These questions include: (a) which essays are…

This research examines how public school principals in seven U.S. states perceive the proficiency exam they must take and pass in order to evaluate their teachers. The test is centered on the states' primary teaching evaluation system, which is based on Charlotte Danielson's Framework for Teaching. An online survey was developed and 832 out of…

Teacher-evaluation has become a key measure for determining teacher effectiveness and the examination of the evaluation process at early childhood levels is a critical area of study. Responding to federal policies, many states have reformed their teacher evaluation systems. One of the more recent developments in state policy is the inclusion of…

Background: Readability tests are indicators that measure how easy a document can be read and understood. Simple, but very often ignored, readability statistics cannot only provide information about the level of difficulty of the readability of particular documents but also can increase an evaluator's credibility. Purpose: The purpose of this…

Evaluators face a number of challenges in using student assessment data, given varying state and federal accountability requirements. Approaches to measuring student progress and specific characteristics of state assessment systems influence how data can be used. The continuous changes to state assessment systems and data create a significant…

In response to Lissitz and Samuelsen (2007), the author reconstructs the historical arguments for the more comprehensive unitary concept of validity and the principles of scientific inquiry underlying it. Her response is organized in terms of four questions: (a) How did validity in educational measurement come to be conceptualized as unitary, and…

A literature review and a proposed means of measuring face validity, a test's appearance of being valid, are presented. Empirical evidence from examinees' perceptions of a college entrance examination support the reliability of measuring face validity. (GDC)

A behavioral assessment system for scoring the behaviors of parents and coaches at youth sports games is described within this paper. The Youth Sports Behavior Assessment System (YSBAS) contains nine behavioral categories describing behaviors commonly seen during youth sports. The developmental process of YSBAS and the observer-training program…

The increasing availability and performance of computer-based testing has prompted more research on the automatic assessment of language and speaking proficiency. In this investigation, we evaluated the feasibility of using an off-the-shelf speech-recognition system for scoring speaking prompts from the LanguEdge field test of 2002. We first…

Using videos of physiotherapy students, compared two assessment forms for validity and reliability (the first currently used by an academic program and the second developed from practitioners' perceptions of competence). Also investigated effects of training on assessment decisions. Found wide differences in individual ability to assess students…