The purpose of this synthesis study was to assess the reliability and validity of the Draw-A-Scientist Test (DAST) and its variations across multiple studies, aiming to understand limitations and propose modifications for future application within and beyond the science domain. Given the existence of multiple DAST versions, this study quantified…

Purpose: Intentional communication acts, or purposefully directed vocalizations and gestures, are particularly difficult for infants at elevated likelihood for eventual diagnosis of autism. The ability to measure and track intentional communication in infancy thus has the potential to aid early identification and intervention efforts. This study…

The "consensual assessment technique" (CAT) is a reliable and valid method to measure (product) creativity and often considered "the" gold standard of creativity assessment. The reliability measure traditionally applied in CAT studies--inter-rater reliability--cannot capture time-sampling error, which is a particular relevant…

Teacher preparation programs are increasingly expected to use data on preservice teacher (PST) skills to drive program improvement and provide targeted supports. Observational ratings are especially vital, but also prone to measurement issues. Scores may be influenced by factors unrelated to PSTs' instructional skills, including rater standards.…

This study reviews existing research on the use of large language models (LLMs) for automatic item generation (AIG). We performed a comprehensive literature search across seven research databases, selected studies based on predefined criteria, and summarized 60 relevant studies that employed LLMs in the AIG process. We identified the most commonly…

In recent years, to better face university paths, the first approaches to the labor market, and then the actual university-to-work transition, university students are asked to have broader skills, such as the ability to network, to be involved in career-related issues, and to explore the characteristics of occupations as much as personal ones.…

The purpose of the study was to develop the Teachers' Mental Toughness Questionnaire (TMTQ). The questionnaire was developed in six stages: item generation, content validity, exploratory factor analysis, reliability tests, convergent validity tests, and discriminant validity. The factor analysis indicates that it measures six factors: team,…

Although AI transformer models have demonstrated notable capability in automated scoring, it is difficult to examine how and why these models fall short in scoring some responses. This study investigated how transformer models' language processing and quantification processes can be leveraged to enhance the accuracy of automated scoring. Automated…

Algorithmic bias is a pressing concern in educational data mining (EDM), as it risks amplifying inequities in learning outcomes. The Area Between ROC Curves (ABROCA) metric is frequently used to measure discrepancies in model performance across demographic groups to quantify overall model fairness. However, its skewed distribution--especially when…

In this paper, the author will develop a more comprehensive notion of truth, one that goes beyond the epistemological correspondence theory, and the author will argue for the importance of authentication as a crucial extension of truth, especially in a posttruth climate. Hannah Arendt observes, "facts need testimony to be remembered and…

This paper aims to synthesize measures of assessment literacy in higher education by forging a connection between two research domains: educational assessment and psychometrics. It begins with a systematic review of assessment literacy measures within the context of higher education published within the last ten years. AL measures, including tests…

This study investigates the estimation of classification consistency and accuracy indices for composite summed and theta scores within the SS-MIRT framework, using five popular approaches, including the Lee, Rudner, Guo, Bayesian EAP, and Bayesian MCMC approaches. The procedures are illustrated through analysis of two real datasets and further…

This study considers the estimation of marginal reliability and conditional accuracy measures using a generalized recursion procedure with several IRT-based ability and score estimators. The estimators include MLE, TCC, and EAP abilities, and corresponding test scores obtained with different weightings of the item scores. We consider reliability…

Assuring the quality of education is a concern of learning institutions. To do so, it is necessary to have assertive learning management, with consistent data on students' outcomes. This research provides associate deans and researchers, a roadmap with which to gather evidence to improve the quality of open-ended assessments. Based on statistical…

Evaluation of child-friendly school (CFS) policies is essential to determine the achievements of school efforts in reducing violence cases. This research aims to proving the reliability and validity of CFS policy evaluation instruments in elementary schools with different locations. This investigation uses the Context Input Process Product (CIPP)…