This study reviews existing research on the use of large language models (LLMs) for automatic item generation (AIG). We performed a comprehensive literature search across seven research databases, selected studies based on predefined criteria, and summarized 60 relevant studies that employed LLMs in the AIG process. We identified the most commonly…

As survey research in second language acquisition grows in popularity, the adherence to best practices associated with questionnaire quality is critical for a better understanding of factors that influence second language (L2) development. To ensure that a self-report scale targets the construct of interest and does it consistently and accurately,…

The "Rating Scale of Impairment" (RSI; Goldstein & Naglieri, 2016b) is a norm-referenced measure of functional impairment. The RSI measures impairment in six domains, as well as overall impairment, based in part on the International Classification of Functioning, Disability, and Health. Functional impairment, as defined by the ICF…

Literature review shows that the development process of an achievement test is mainly investigated in dissertations. Moreover, preparing a form that will shed light on developing an achievement test is expected to guide those who will administer the test. In this line, the current study aims to create an "Achievement Test Development Process…

Adult learners are attracted to learning opportunities (e.g. course offers) which seem promising in terms of allowing them to match their choices to their own perceived predispositions. To find out more about their personal learning style, some adult learners may fill in a questionnaire designed by researchers who aim (and claim) to enable both…

This research brief summarizes the available reliability and validity data available in, but spread throughout, a number of General Educational Development (GED) Testing Service publications. A section on reliability discusses how to determine reliability of a test's scores and two ways of assessing the reliability of a test--internal consistency…

Details the steps followed in the development of the Basic Skills Word List. (FL)

Notes that significantly different results frequently exist between tests that purport to measure the same skill when the same child is tested on both instruments. Considers discrepancies related to examinee, examiner, examinee-examiner interactions, environment, and psychometric characteristics of the tests employed. Cites 10 major psychometric…

Argues that, in determining if a standardized test task and a classroom performance task are the same, educators should carefully analyze the tasks in terms of the input modality (stimulus) and the output modality (response). (FL)

Extensive research efforts in the past decade have measured academic proficiency and progress in both theory and practice of large-scale writing assessment. The following major situational variables were identified in these investigations: (1) topic variables such as writing prompts, subject matter, mode of discourse, length, and rhetorical…

In this paper 45 item-writing rules for multiple-choice tests presented in textbooks on educational measurement in a previous study are identified. The current study presents a quantitative review of the literature with respect to the empirical and theoretical evaluation of these principles of item-writing. Fifty-six studies that addressed at…

Order Analysis is a multidimensional scaling (MDS) technique for determining order among items. This paper reviews articles by different authors describing various components of ordering theory. A common nomenclature is constructed to link together the various ideas and is applied to a fairly simple set of data. Topics discussed include a more…

Discusses problems with the current trend in using behavior trait-based criteria to assess English-as-a-Second-Language productivity skills, and describes alternatives to such testing that involve the matching of linguistic tasks against nonlinguistic criteria. (Author/CB)

Literature related to the multiple true-false (MTF) item format is reviewed. Each answer cluster of a MTF item may have several true items and the correctness of each is judged independently. MTF tests appear efficient and reliable, although they are a bit harder than multiple choice items for examinees. (SLD)

Most seasoned test developers recognize the importance of thoughtful decision making when constructing a test. Unfortunately, many classroom achievement tests are created by novice test developed who have not received sufficient instruction in item writing (G. Gulliksen, 1986; R. J. Stiggins, 1991). The result is often a test that is poorly…