Automatic item generation may supply many items instantly and efficiently to assessment and learning environments. Yet, the evaluation of item quality persists to be a bottleneck for deploying generated items in learning and assessment settings. In this study, we investigated the utility of using large-language models, specifically Llama 3-8B, for…

Computer-based assessment (CBA) is a top-rated tool for conducting assessments, mapping learning outcomes, and selecting new candidates. Research that examines the development and use of CBA is also increasing from year to year, so without bibliometric analysis, it would be quite challenging to keep up with all of these studies. This study aims to…

This article discusses critical methodological design decisions for collecting, interpreting, and synthesizing empirical evidence during the design, deployment, and operational quality-control phases for automated scoring systems. The discussion is inspired by work on operational large-scale systems for automated essay scoring but many of the…

Companies purchase computer systems to make their processes more efficient through automation. Some academic medical centers (AMC) have purchased computer systems for their institutional review boards (IRB) to increase efficiency and compliance with regulations. IRB computer systems are expensive to purchase, deploy, and maintain. An AMC should…

The bicentenary in 2011 of the Luddite Revolt prompts us to ask "what would Ned Ludd think of today's automated styles of distance education?" He would no doubt echo the common criticism that educational technologies create an impersonal style of teaching and learning, and devalue the teacher. He would probably agree that online methods…

Automated scoring of constructed responses is already operational in several testing programmes. However, as the methodology matures and the demand for the utilisation of constructed responses increases, the volume of automated scoring is likely to increase at a fast pace. Quality assurance and control of the scoring process will likely be more…

Automated assessment is an emerging innovation in educational praxis, however its pedagogical potential is not fully utilised in Australia, particularly regarding automated essay grading. The rationale for this research is that the usage of automated assessment currently lags behind the capacity that the technology provides, thus restricting the…

Discusses the quality controls imposed on text and image data that is currently being converted from paper to digital images by the Patent and Trademark Office. The methods of inspection used on text and on images are described, and the quality of the data delivered thus far is discussed. (CLB)

As automated scoring of complex constructed-response examinations reaches operational status, the process of evaluating the quality of resultant scores, particularly in contrast to scores of expert human graders, becomes as complex as the data itself. Using a vignette from the Architectural Registration Examination (ARE), this article explores the…

Conducted survey in 32 psychiatric facilities to determine the types and sources of information used in care review in order to develop a meaningful minimum data set for quality assurance purposes. Found that the adequacy of medical record information for review and the feasibility of automating it depended on the nature of the review task.…

The paper addresses the potential of informatics to patient care quality assurance curricula, focusing on (1) terminology and developments related to quality of care evaluations; (2) criticisms of traditional approaches; (3) limitations of existing data sources for quality assurance in dentistry; and (4) quality assurance considerations in…