This study reviews existing research on the use of large language models (LLMs) for automatic item generation (AIG). We performed a comprehensive literature search across seven research databases, selected studies based on predefined criteria, and summarized 60 relevant studies that employed LLMs in the AIG process. We identified the most commonly…

Purpose: The goal of the current study was to conduct a substantive validity review of four autism knowledge assessments with prior psychometric support (Gillespie-Lynch in J Autism and Dev Disord 45(8):2553-2566, 2015; Harrison in J Autism and Dev Disord 47(10):3281-3295, 2017; McClain in J Autism and Dev Disord 50(3):998-1006, 2020; McMahon…

The growing diversity among test takers in second or foreign language (L2) assessments makes the importance of fairness front and center. This systematic review aimed to examine how fairness in L2 assessments was evaluated through differential item functioning (DIF) analysis. A total of 83 articles from 27 journals were included in a systematic…

As the use of process data in large-scale educational assessments is becoming more common, it is clear that data on examinees' test-taking behaviors can illuminate their performance, and can have crucial ramifications concerning assessments' validity. A thorough review of the literature in the field may inform researchers and practitioners of…

As survey research in second language acquisition grows in popularity, the adherence to best practices associated with questionnaire quality is critical for a better understanding of factors that influence second language (L2) development. To ensure that a self-report scale targets the construct of interest and does it consistently and accurately,…

The New Ecological Paradigm (NEP) scale adapted for use with children (NEP-C) is one of the most frequently used measures of children's environmental beliefs. Though widely utilized, the limitations of the NEP-C instrument are often overlooked. Based on a systematic synthesis of existing literature examining the NEP-C, we argue that the scale…

The "Rating Scale of Impairment" (RSI; Goldstein & Naglieri, 2016b) is a norm-referenced measure of functional impairment. The RSI measures impairment in six domains, as well as overall impairment, based in part on the International Classification of Functioning, Disability, and Health. Functional impairment, as defined by the ICF…

Adult learners are attracted to learning opportunities (e.g. course offers) which seem promising in terms of allowing them to match their choices to their own perceived predispositions. To find out more about their personal learning style, some adult learners may fill in a questionnaire designed by researchers who aim (and claim) to enable both…

This paper details a systematic literature review identifying problems in extant research relating to teachers' attitudes toward reporting child sexual abuse and offers a model for new attitude scale development and testing. Scale development comprised a five-phase process grounded in contemporary attitude theories, including (a) developing the…

Computerized adaptive testing in personality assessment can improve efficiency by significantly reducing the number of items administered to answer an assessment question. Two approaches have been explored for adaptive testing in computerized personality assessment: item response theory and the countdown method. In this article, the authors…

As test standards and research literature in general indicate, definitions of test bias and item bias vary considerably, as do the results of existing methods of identifying biased items. The situation is further complicated by issues of content, context, construct, and criterion. In achievement tests, for example, content validity may impose…

The test-item bias literature is summarized, emphasizing the conceptual basis for bias detection methods and the technical issues involved in choosing among methods. It describes both judgmental and statistical methods for identifying biased items, and discusses the reconciliation of these two types of evidence. (Author/BW)

This research brief summarizes the available reliability and validity data available in, but spread throughout, a number of General Educational Development (GED) Testing Service publications. A section on reliability discusses how to determine reliability of a test's scores and two ways of assessing the reliability of a test--internal consistency…

The central idea in building and maintaining an item bank is to calibrate all the items onto a "common variable." The arithmetic involved in the calibration process is presented. It is recommended that an analysis of fit be done in every application to verify that the estimates of item difficulties are in fact sample-free. These…

A critical review is presented of research conducted during the past 20 years on multiple-choice tests of achievement and aptitude. The design and use of multiple-choice tests is emphasized, but information concerning the socioeducational implications of relying on such tests is also included. (Author/CB)