Background: Multiple-choice item (MCI) assessments are burdensome for instructors to develop. Artificial intelligence (AI, e.g., ChatGPT) can streamline the process without sacrificing quality. The quality of AI-generated MCIs and human experts is comparable. However, whether the quality of AI-generated MCIs is equally good across various domain-…

Despite their drawback, multiple-choice questions are an enduring feature in instruction because they can be answered more rapidly than open response questions and they are easily scored. However, it can be difficult to generate good incorrect choices (called "distractors"). We designed an algorithm to generate distractors from a…

Developing effective assessments is a critical component of quality instruction. Assessments are effective when they are well-aligned with the learning outcomes, can confirm that all intended learning outcomes are attained, and their obtained grades are accurately reflecting the level of student achievement. Developing effective assessments is not…

As a high-stakes standardized test, IELTS is expected to have comparable forms of test papers so that test takers from different test administration on different dates receive comparable test scores. Therefore, this study examined the text difficulty and task characteristics of four parallel academic IELTS reading tests to reveal to what extent…

We present a tool for the creation and curation of C-tests. C-tests are an established tool in language proficiency testing and language learning. They require examinees to complete a text in which the second half of every second word is replaced by a gap. We support teachers and test designers in creating such tests through a web-based system…

Assessment is a useful process as it provides various stakeholders (e.g., teachers, parents, government, employers) with information about students' competence in a particular subject area. However, for the information generated by assessment to be useful, it needs to support valid inferences. One factor that can undermine the validity of…

Physics literacy plays a crucial part in global technological development as several aspects of science and technology apply concepts and principles of physics in their operations. However, the acquisition of scientific literacy in physics in our society today is not encouraging enough to the desirable standard. Therefore, this study focuses on…

The "TextEvaluator"® text analysis tool is a fully automated text complexity evaluation tool designed to help teachers, curriculum specialists, textbook publishers, and test developers select texts that are consistent with the text complexity guidelines specified in the Common Core State Standards.This paper documents the procedure used…

This paper considers whether the sources of linguistic complexity presented within texts targeted at 1st-grade readers have increased, decreased, or held steady over the 52-year period from 1962 to 2013. A collection of more than 450 texts is examined. All texts were selected from Grade 1 textbooks published by Scott Foresman during the targeted…

These proceedings contain the papers of the 16th International Conference on Cognition and Exploratory Learning in the Digital Age (CELDA 2019), held during November 7-9, 2019, which has been organized by the International Association for Development of the Information Society (IADIS) and co-organised by University Degli Studi di Cagliari, Italy.…

This study focuses on the relationship between item modeling and evidence-centered design (ECD); it considers how an appropriately generalized item modeling software tool can support systematic identification and exploitation of task-model variables, and then examines the feasibility of this goal, using linear-equation items as a test case. The…

Two alternatives to traditional item analysis and reliability estimation procedures are considered for determining the difficulty, discrimination, and reliability of optional items on essay and other tests. A computer program to compute these measures is described, and illustrations are given. (SLD)

Computer adaptive testing is the study of scoring tests and questions based on assumptions concerning the mathematical relationship between examinees' ability and the examinees' responses. Adaptive student tests, which are based on item response theory (IRT), have many advantages over conventional tests. We use the least square method, a…

This study investigated the characteristics of Computer-based Case Simulations (CCS) that may be associated with case difficulty. Difficulty was defined as the average rating by physicians of examinee performance on a nine-point scale or the passing rate on the cases. Two data sets were used, one from an administration of 18 cases to 201 medical…

The hardware and software system used to create the National Opinion Research Center/Center for Research on Evaluation, Standards, and Student Testing (NORC/CRESST) item databases and test booklets for the 12th-grade science assessment are described. A general description of the capabilities of the system is given, with some specific information…