This study reviews existing research on the use of large language models (LLMs) for automatic item generation (AIG). We performed a comprehensive literature search across seven research databases, selected studies based on predefined criteria, and summarized 60 relevant studies that employed LLMs in the AIG process. We identified the most commonly…

The growing diversity among test takers in second or foreign language (L2) assessments makes the importance of fairness front and center. This systematic review aimed to examine how fairness in L2 assessments was evaluated through differential item functioning (DIF) analysis. A total of 83 articles from 27 journals were included in a systematic…

Knowledge is an important predictor and outcome of learning and development. Its measurement is challenged by the fact that knowledge can be integrated and homogeneous, or fragmented and heterogeneous, which can change through learning. These characteristics of knowledge are at odds with current standards for test development, demanding a high…

GCSE examinations (taken by students aged 16 years in England) are not intended to be speeded (i.e. to be partly a test of how quickly students can answer questions). However, there has been little research exploring this. The aim of this research was to explore the speededness of past GCSE written examinations, using only the data from scored…

Computational thinking (CT) is regarded as an essential twenty-first century competency and it is already embedded in K-12 curricula across the globe. However, research on assessing CT has lagged, with few assessments being implemented and validated. Moreover, there is a lack of systematic grouping of CT assessments. This scoping review examines…

The purpose of test directions is to familiarize examinees with a test so that they respond to items in the manner intended. However, changes in educational measurement as well as the U.S. student population present new challenges to test directions and increase the impact that differential familiarity could have on the validity of test score…

Argues that multiple choice tests can be effective only if the items are written in a format suitable for testing the mastery of specific instructional objectives. Proposes the use of nonrestrictive test items and cites examples of such items. (FL)

Results of 96 theoretical/empirical studies were reviewed to see if they support a taxonomy of 43 rules for writing multiple-choice test items. The taxonomy is the result of an analysis of 46 textbooks dealing with multiple-choice item writing. For nearly half of the rules, no research was found. (SLD)

A comprehensive review is given of evidence, with a bearing on the recommendation to avoid use of complex multiple choice (CMC) items. Avoiding Type K items (four primary responses and five secondary choices) seems warranted, but evidence against CMC in general is less clear. (SLD)

Analysis of literature on oral assessment in college instruction identified six dimensions: primary content type; interaction between examiner and learner; authenticity of assessment task; structure of assessment task; examiner; and orality (extent to which knowledge is tested orally). These help in understanding the nature of oral assessment and…

Argues that multiple-choice tests can be effective only if the items are written in a format suitable for testing the mastery of specific instructional objectives. Proposes the use of nonrestrictive test items and cites examples of such items. (FL)

Summarizes item review in the development of criterion-referenced tests, including logical item review, which examines the match between instructional intent and the items; empirical item review, which examines response patterns; traditional item review; and instructional sensitivity of test items. Twenty-eight references are listed. (MBR)

The second of three sequels to the booklet "Student Assessment," this booklet begins by describing and giving examples of three different forms that short-answer questions can take: (1) completion items; (2) unique-answer questions; and (3) open short-answer questions. Guidelines are then provided for deciding which type of question to…

A meta-analysis of the difficulty and discrimination of the "none-of-the-above" (NOTA) test option was conducted with 12 articles (20 effect sizes) for difficulty and 7 studies (11 effect sizes) for discrimination. Findings indicate that using the NOTA option does not result in items of lesser quality. (SLD)

The first of three sequels to the booklet "Student Assessment," this booklet begins by describing and providing examples of four different forms that objective questions can take: (1) conventional multiple choice questions; (2) true/false questions; (3) assertion/reason items; and (4) matching items. Guidance is offered on how to decide which type…