Great opportunities for language testing practitioners are enabled through language program administration. Local language tests lend themselves to multiple purposes--for placement and diagnosis, as a means of tracking progress, and as a contribution to program evaluation and revision. Administrative choices, especially those involving a test, are…

Purpose: The purpose of the self-study is two-fold: first, to aid in redressing the lack of attention given to the professional development i.e., the building of practical or work-related knowledge of examination invigilators and second, to forward the idea that engaging the examination invigilation process reflectively is an effective form of…

Assessment plays an important role in learning and Multiple Choice Questions (MCQs) are quite popular in large-scale evaluations. Technology-enabled learning necessitates a smart assessment. Therefore, automatic MCQ generation became increasingly popular in the last two decades. Despite a large amount of research effort, system generated MCQs are…

This technical assistance paper provides guidance to individuals with test administration responsibilities in adult education programs. These policies apply to the approved assessments that programs may use to report educational gains in compliance with the National Reporting System (NRS) requirements. The NRS is the accountability system for the…

In 2020, because of the COVID-19 pandemic the higher education sector, in the United Kingdom and internationally, transitioned to online assessment, at a speed and scale which might have been unimaginable under normal circumstances. The priority in the sector was to ensure that fundamental principles of assessment, including integrity, were…

Evaluation professionals need to be nimble and innovative in their approaches in order to be relevant and provide useful evidence to decision-makers, stakeholders, and society in the crowded public policy landscape. In this article, we offer serious games as a method that can be employed by evaluators to address three persisting challenges in…

To begin statistical analysis, Bayesians quantify their confidence in modeling hypotheses with priors. A prior describes the probability of a certain modeling hypothesis apart from the data. Bayesians should be able to defend their choice of prior to a skeptical audience. Collaboration between evaluators and stakeholders could make their choices…

While much of the work in language testing is concerned with constructing quality tests in order to measure language knowledge in reliable and valid ways, there has been a significant movement in language testing research that examines tests in the context of their use in education and society. This line of research exits from the notion that…

E-rater[R] is an automated essay scoring system that uses natural language processing techniques to extract features from essays and to model statistically human holistic ratings. Educational Testing Service has investigated the use of e-rater, in conjunction with human ratings, to score one of the two writing tasks on the TOEFL-iBT[R] writing…

The competencies required for conducting assessment in the educational context have recently been reformulated in view of social constructivist perspectives and the acknowledgement of the role of classroom assessment in promoting learning. These changes have impacted the knowledge base language assessors need to obtain, and hence the contents of…

As a result of investigations showing that communication problems can be a significant contributing factor to major aviation accidents, the International Civil Aviation Organization (ICAO) has established a set of Language Proficiency Requirements. All pilots and air traffic controllers engaged in international aviation must be certified by their…

Many approaches to standard-setting use item calibration and student score estimation results to structure panelists' tasks. However, this requires collecting standard-setting judgments after the item analysis results are available. The Scoring Guide Alignment approach collects standard-setting judgments during the scoring sessions from teachers…

Policy-making problems being faced in Louisiana regarding the testing of handicapped students within a state mandated minimum competency testing program are dealt with. The decision-making process is complicated by the fact that two groups (staff administering the test and staff administering programs for the handicapped) with differing…

Proposes a process approach to validity that addresses assessment validation in the context of high-stakes assessment. This approach includes a test evaluator or validator who considers the perspectives of five stakeholder groups at four different stages of assessment maturity in relation to six aspects of construct validity. Illustrates each…

In view of social processes, politics and the logistics of conducting evaluations, several implicit assumptions of the field challenged by findings on innovation implementation are reviewed. These studies have major implications for evaluation practice, evaluator roles, the use of evaluation results, and the federal role in evaluation. (Author/CM)