This paper presents improvements made to a computerized adaptive testing (CAT)-based version of the FCI (FCI-CAT) in regards to test security and test efficiency. First, we will discuss measures to enhance test security by controlling for item overexposure, decreasing the risk that respondents may (i) memorize the content of a pretest for use on…

This study examined the development and technical adequacy of the Interventionist Knowledge and Skill Survey (IKSS), a self-assessment used for understanding the degree of knowledge and skills associated with interventionists working within a multitiered system of support (MTSS). A national sample of 600 school psychologists and school counselors…

Many students experience a language-related barrier which may keep them from fully understanding and engaging in complex chemistry content. Although dense and technically rich scientific language is challenging for most students to interpret, it can become a systematic barrier for English Language Learner (ELL) students when it cognitively…

In education, examining students' learning in detail, determining their strengths and weaknesses and giving effective feedback have gained importance over time. The aim of this study is to determine the distribution of students' answers to the reading comprehension achievement test items which were written at different cognitive levels and to…

It is known that students have many misconceptions about concepts related to function. By discovering misconceptions using an appropriate measurement instrument, we can determine what changes we need to make in the real functions curriculum to improve learning outcomes. Therefore, we designed an item bank for measuring conceptual understandings of…

Differential item functioning (DIF) is often used to examine validity evidence of alternate form test accommodations. Unfortunately, traditional approaches for evaluating DIF are prone to selection bias. This article proposes a novel DIF framework that capitalizes on regression discontinuity design analysis to control for selection bias. A…

Response process data have the potential to provide a rich description of test-takers' thinking processes. However, retrieving insights from these data presents a challenge for educational assessments and educational data mining as they are complex and not well annotated. The present study addresses this challenge by developing a computational…

This research is a developmental research aiming at developing a good mathematical test instrument using polytomous responses based on classical and modern theories. This research design uses the Plomp model, which consists of five stages, (1) preliminary investigation, (2) design, (3) realization/construction, (4) revision, and (5) implementation…

This study aims to examine the effects of mixture item response theory (IRT) models on item parameter estimation and classification accuracy under different conditions. The manipulated variables of the simulation study are set as mixture IRT models (Rasch, 2PL, 3PL); sample size (600, 1000); the number of items (10, 30); the number of latent…

In test equating, one critical equating property is the group invariance property which indicates that the equating function used to convert performance on each alternate form to the reporting scale should be the same for various subgroups. To mitigate the impact of disrupted learning on the item parameters during the COVID-19 pandemic, a…

Intervention research in education is sometimes criticized for the use of experimenter developed assessments, especially when these are over aligned with treatment. At the same time, intervention researchers sometimes prefer locally developed assessments because they appear to be more sensitive to treatment effects even when the test is not…

It has become a critical question what the reliability level would be when open-ended questions are used in large-scale selection tests. One of the aims of the present study is to determine what the reliability would be in the event that the answers given by test-takers are scored by experts when open-ended short answer questions are used in…

The aim of this study is to analyse the science questions in terms of visual content in the higher education entrance exams in Turkey. In this context, 1714 questions in total prepared by the Center for Measurement, Selection and Placement (CMSP) between 1999 and 2019 in the fields of Physics (n=631), Chemistry (n=553) and Biology (n=530)…

We used simulation techniques to assess the item-level and familywise Type I error control and power of an IRT item-fit statistic, the "S-X"[superscript 2]. Previous research indicated that the "S-X"[superscript 2] has good Type I error control and decent power, but no previous research examined familywise Type I error control.…

This study aims to examine the potential impacts on repeat examinees' performance by reusing simulation-based items in a high-stakes standardized assessment. We examined change patterns of item scores, ability estimate, score pattern change, response time and compared the performance of repeat examinees who have received repeat items and those who…