The development of a Computerized Adaptive Test (CAT) for operational use begins with several important steps, such as creating a large-size item bank, piloting the items on a sizable and representative sample of examinees, dimensionality assessment of the item bank, and estimation of item parameters. Among these steps, testing the dimensionality…

The Centre for Evaluation and Monitoring (CEM), based in the North of England, recently celebrated its 40th birthday. Arising from an evaluation project at Newcastle University, and a subsequent move to Durham University, it rapidly grew in scope and influence, developing a series of highly regarded school assessments. For a relatively small…

Although many educational and psychological tests are labeled as computerized adaptive test (CAT), not all tests show the same level of adaptivity -- some tests might not have much adaptation because of various constraints imposed by test developers. Researchers have proposed some indices to measure the amount of adaption for an adaptive test.…

Administration of patient-reported outcome measures (PROs), using multidimensional computer adaptive tests (MCATs) has the potential to reduce patient burden, but the efficiency of MCAT depends on the degree to which an individual's responses fit the psychometric properties of the assessment. Assessing patients' symptom burden through the…

Transparency in high-stakes English language assessment has become crucial for ensuring fairness and maintaining assessment validity in language testing. However, our understanding of how transparency is conceptualised and implemented remains fragmented, particularly in relation to stakeholder experiences and technological innovations. This study…

Computerized adaptive testing (CAT) has gained deserved popularity in the administration of educational and professional assessments, but continues to face test security challenges. To ensure sustained quality assurance and testing integrity, it is imperative to establish and maintain multiple stable item pools that are consistent in terms of…

Motivation to read is crucial to improving reading skill. While there is extensive research examining reading motivation among elementary students, with respect to adolescents, research is limited. Employing a person-centered approach can aid in developing a better understanding of adolescent reading motivation and would help address possible…

To solve the attenuation paradox in computerized adaptive testing (CAT), this study proposes an item selection method, the integer programming approach based on real-time test data (IPRD), to improve test efficiency. The IPRD method turns information regarding the ability distribution of the population from real-time test data into feasible test…

In computerized adaptive testing, overexposure of items in the bank is a serious problem and might result in item compromise. We develop an item selection algorithm that utilizes the entire bank well and reduces the overexposure of items. The algorithm is based on collaborative filtering and selects an item in two stages. In the first stage, a set…

The purpose of this study is to describe and evaluate a multilingual automated essay scoring (AES) system for grading essays in three languages. Two different sentence embedding models were evaluated within the AES system, multilingual BERT (mBERT) and language-agnostic BERT sentence embedding (LaBSE). German, Italian, and Czech essays were…

ETS experts are exploring and defining the standards for responsible AI use in assessments. A comprehensive framework and principles will be unveiled in the coming months. In the meantime, this document outlines the critical areas these standards will encompass, including the principles of: (1) Fairness and bias mitigation; (2) Privacy and…

This study investigated the impact of scoring polytomous items later on measurement precision, classification accuracy, and test security in mixed-format adaptive testing. Utilizing the shadow test approach, a simulation study was conducted across various test designs, lengths, number and location of polytomous item. Results showed that while…

In educational tests, a group of questions related to a shared stimulus is called a testlet (e.g., a reading passage with multiple related questions). Use of testlets is very common in educational tests. Additionally, computerized adaptive testing (CAT) is a mode of testing where the test forms are created in real time tailoring to the test…

Adaptive measurement of change (AMC) is a psychometric method for measuring intra-individual change on one or more latent traits across testing occasions. Three hypothesis tests--a Z test, likelihood ratio test, and score ratio index--have demonstrated desirable statistical properties in this context, including low false positive rates and high…

Background Study: Assessing learners' inquiry-based skills is challenging as social, political, and technological dimensions must be considered. The advanced development of artificial intelligence (AI) makes it possible to address these challenges and shape the next generation of science education. Objectives: The present study evaluated the SSI…