Multiple choice questions are traditionally expensive to produce. Recent advances in large language models (LLMs) have led to fine-tuned LLMs that generate questions competitive with human-authored questions. However, the relative capabilities of ChatGPT-family models have not yet been established for this task. We present a carefully-controlled…

Presents minimization and maximization models for parallel test construction under constraints. The minimization model constructs weakly and strongly parallel tests of minimum length, while the maximization model constructs weakly and strongly parallel tests with maximum test reliability. (Author/SLD)

This study is a critical evaluation of the roles for coding and scoring of missing responses to multiple-choice items in educational tests. The focus is on tests in which the test-takers have little or no motivation; in such tests omitting and not reaching (as classified by the currently adopted operational rules) is quite frequent. Data from the…

It is unrealistic to suppose that standard item response theory (IRT) models will be appropriate for all new and currently considered computer-based tests. In addition to developing new models, researchers will need to give some attention to the possibility of constructing and analyzing new tests without the aid of strong models. Computerized…

Describes the academic context in which the "French CAT" was created and trialed and gives a detailed consideration of the test presentation platform and operating algorithms. Finally, the article evaluates the first administration of the test and discusses its reliability and validity as a placement instrument for first-year Australian…

Items with the highest discrimination parameter values in a logistic item response theory (IRT) model do not necessarily give maximum information. This paper shows which discrimination parameter values (as a function of the guessing parameter and the distance between person ability and item difficulty) give maximum information for the…

Four discussions of test construction based on item response theory (IRT) are presented. The first discussion, "Test Design as Model Building in Mathematical Programming" (T. J. J. M. Theunissen), presents test design as a decision process under certainty. A natural way of modeling this process leads to mathematical programming. General…

This article gives a comprehensive description of a theory for efficient assessment of knowledge. The essential concept is that the knowledge state of a subject, with regard to a specified field of information, can be represented by a particular subset of problems that the subject is capable of solving. (SLD)

A paradigm for the recalibration of teacher-made assessment that assesses and evaluates in one operation is formulated. The effort to make the classroom the primary source of educational research activity is contingent on redefining educational research as empirical and not experimental. This emphasizes that the empirical analysis of instructional…