In this article, we systematize the factors influencing performance and feasibility of automatic content scoring methods for short text responses. We argue that performance (i.e., how well an automatic system agrees with human judgments) mainly depends on the linguistic variance seen in the responses and that this variance is indirectly influenced…

Machine learning methods can be used to reduce the manual workload in exam grading, making it possible for teachers to spend more time on other tasks. However, when it comes to grading exams, fully eliminating manual work is not yet possible even with very accurate automated grading, as any grading mistakes could have significant consequences for…

The upgrade educational information technology triggered by COVID-19 has shaped a new educational order and new educational forms. As a result, traditional educational measurement is now facing a systematic transformation, that is, from the Assessment of Learning (AoL) to Assessment for Learning (AfL), and finally to Assessment as Learning (AaL).…

This book explores the transformative role of artificial intelligence in educational assessment, catering to researchers, educators, administrators, policymakers, and technologists involved in shaping the future of education. It delves into the foundations of AI-assisted assessment, innovative question types and formats, data analysis techniques,…

This paper presents the concepts and observations in the author's keynote address at the May 2019 "Opportunity versus Challenge: Exploring Usage of Log-File and Process Data in International Large-Scale Assessments" conference in Dublin, Ireland. This paper recaps briefly some key points that emerged at the December 2018 ETS symposium on…

A common approach to increase test security in higher educational high-stakes testing is the use of different test forms with identical items but different item orders. The effects of such varied item orders are relatively well studied, but findings have generally been mixed. When multiple test forms with different item orders are used, we argue…

College and university admissions officers are the gatekeepers who decide which individuals may join the "country club" of higher education. As such, the decisions they make have a significant impact on the lives of individuals, especially those who are willing to travel across the globe and risk living in an unfamiliar country in order…

Much has been done on assessment literacy (AL) components and job demand-resources (JD-R). However, an interdisciplinary look at AL components as the predictors of JD-R and its possible consequences for the engagement and burnout of teachers' assessment performance has been neglected. To fill this gap, the present study explored this issue in the…

Drawing on duoethnography, the teacher researchers in the present study interacted with the relevant literature, engaged in dialogs, and shared artifacts to examine their knowledge, beliefs, and practices regarding fairness and justice considerations in technology-enhanced language classroom assessment. Under the domain of knowledge, they…

Testing and data integrity on statewide assessments is defined as the establishment of a comprehensive set of policies and procedures for: (1) the proper preparation of students; (2) the management and administration of the test(s) that will lead to accurate and appropriate reporting of assessment results; and (3) maintaining the security of…

Three ethical-legal case incidents involving computer-assisted psychodiagnostic assessment in the schools are presented. Each case is analyzed in terms of applicable laws, ethical codes, and standards for professional practice. (Author)

The increased use of computer-based and Internet-delivered testing has raised a number of ethical and legal issues. The International Test Commission's (this issue) Guidelines for Computer-Based and Internet-Delivered Testing represent the most recent attempt to provide test users, publishers, and developers with guidance regarding the appropriate…

Describes the development and testing of a structured ethical integrity interview that can be used in the selection of applicants. Examines directions for future testing and study. (SR)

Final-year Ontario medical students (n=88) took a 4-station objective structured clinical examination (OSCE) using standardized patients and involving decisions to forgo life-sustaining treatment. Performance was scored on a checklist of behaviors unique to each case. Results indicated that because of low reliability, the OSCE is not a feasible…

The Perry scheme of intellectual and ethical development has become widely used in a range of academic disciplines and such areas as career training and faculty consultation. However, current measurement techniques for the scheme, whether interview format or paper and pencil measures, do not adequately address issues related to assessing cognitive…