Effective educational measurement relies heavily on the curation of well-designed item pools. However, item calibration is time consuming and costly, requiring a sufficient number of respondents to estimate the psychometric properties of items. In this study, we explore the potential of six different large language models (LLMs; GPT-3.5, GPT-4,…

We explored three approaches to resolving or re-scoring constructed-response items in mixed-format assessments: rater agreement, person fit, and targeted double scoring (TDS). We used a simulation study to consider how the three approaches impact the psychometric properties of student achievement estimates, with an emphasis on person fit. We found…

The fit of an item response model is typically conceptualized as whether a given model could have generated the data. We advocate for an alternative view of fit, "predictive fit", based on the model's ability to predict new data. We derive two predictive fit metrics for item response models that assess how well an estimated item response…

Introduction: Self-regulation is the ability to adequately plan and manage one's own behavior in a flexible manner. It is a predictor of well-being, health, academic performance, among others. The psychometric characterization of the Self-Regulation Questionnaire-Abbreviated (CAR-abr.) composed of 17 items is presented. A versatile instrument,…

Despite the increasing demand for diagnostic information, observed subscores have been often reported to lack adequate psychometric qualities such as reliability, distinctiveness, and validity. Therefore, several statistical techniques based on CTT and IRT frameworks have been proposed to improve the quality of subscores. More recently, DCM has…

Analyzing heterogeneous treatment effects (HTEs) plays a crucial role in understanding the impacts of educational interventions. A standard practice for HTE analysis is to examine interactions between treatment status and preintervention participant characteristics, such as pretest scores, to identify how different groups respond to treatment.…

The impact of teachers' beliefs on the implementation and effectiveness of Social and Emotional Learning (SEL) programs underscores the essential need for reliable measures of these beliefs. This study aims to explore and validate the psychometric properties of the Teacher Social and Emotional Learning Beliefs Scale (TSELBS) within the Chinese…

Analyzing heterogeneous treatment effects (HTE) plays a crucial role in understanding the impacts of educational interventions. A standard practice for HTE analysis is to examine interactions between treatment status and pre-intervention participant characteristics, such as pretest scores, to identify how different groups respond to treatment.…

The Digital Stress Scale (DSS) is used to measure digital stress, which is the perceived stress and anxiety associated with social media use. In this study, the Chinese version of the DSS was validated using a sample of 721 Chinese college students, 321 males and 400 females (KMO = 0.923; Bartlett = 5,058.492, p < 0.001). Confirmatory factor…

The relationship between latent trait and test-taking speed is an important area of study in assessment research. In addition to contributions of such studies to psychometrics, the factors that affect both ability and speed have implications for test development and have policy consequences especially if the tests are high stakes. This study…

The purpose of this study is to develop Education Value Perception Scale (EVPS) based on Bronfenbrenner's Ecological Theory and to investigate its psychometric properties according to Classical Test Theory (CTT) and Item Response Theory (IRT). The data were collected from 2872 secondary school students by stratified purposeful sampling method.…

The present investigation explored the psychometric validity of the five-item Gratitude Questionnaire (GQ-5) using a construct validation approach. Concerning within-network construct validity, results of item response theory (IRT) analysis via graded response model (GRM) showed that this scale could not discriminate individuals who score high in…

This study reports an analysis of the Force Concept Inventory (FCI) using item response curves (IRC)--the fraction of students selecting each response to an item as a function of their total score. Three large samples (N = 9606, 4360, and 1439) of calculus-based physics students were analyzed. These were drawn from three land-grant institutions…

Critical thinking has been identified as a crucial general skill contributing to academic and career success in the twenty-first century. With the increasing demands of the modern workplace and a global trend of accountability in higher education, educators and employers pay great attention to the development of students' critical thinking skills…

Studies have shown that item difficulty can vary significantly based on the context of an item within a test form. In particular, item position may be associated with practice and fatigue effects that influence item parameter estimation. The purpose of this research was to examine the relevance of item position specifically for assessments used in…