Automated scoring is a current hot topic in creativity research. However, most research has focused on the English language and popular verbal creative thinking tasks, such as the alternate uses task. Therefore, in this study, we present a large language model approach for automated scoring of a scientific creative thinking task that assesses…

Unit-weight sum scores (UWSSs) are routinely used as estimates of factor scores on the basis of solutions obtained with the nonlinear exploratory factor analysis (EFA) model for ordered-categorical responses. Theoretically, this practice results in a loss of information and accuracy, and is expected to lead to biased estimates. However, the…

Job automation is a topical issue in a technology-driven labor market. However, greater amounts of human capital (e.g., often measured by education, and information-processing skills, including adult literacy) are linked with job security. A knowledgeable and skilled labor force better resists unemployment and/or rebounds from job disruption…

Job automation is a topical issue in a technology-driven labor market. However, greater amounts of human capital (e.g., often measured by education, and information-processing skills, including adult literacy) are linked with job security. A knowledgeable and skilled labor force better resists unemployment and/or rebounds from job disruption…

Agreement statistics and measures of prediction accuracy are often used to assess the quality of two measures of a construct. Agreement statistics are appropriate for measures that are supposed to be interchangeable, whereas prediction accuracy statistics are appropriate for situations where one variable is the target and the other variables are…

Teachers must be able to monitor students' behavior and identify valid cues in order to draw conclusions about students' actual engagement in learning activities. Teacher training can support (inexperienced) teachers in developing these skills by using videotaped teaching to highlight which indicators should be considered. However, this supposes…

In recognition of the capability of text-mining models to quantify aspects of language use, some creativity researchers have adopted text-mining models as a mechanism to objectively and efficiently score the Originality of open-ended responses to verbal divergent thinking tasks. With the increasing use of text-mining models in divergent thinking…

Despite several advantages of online education, lack of effective student-instructor interaction, especially when students need timely help, poses significant pedagogical challenges. Motivated by this, we address the problems of automatically identifying posts that express confusion or urgency from Massive Open Online Course (MOOC) forums. To this…

Automatic Speech Scoring (ASS) has increasingly become a useful tool in oral proficiency testing for Second Language (L2) learners. However, limited studies investigate the alignment of ASS indices with the Complexity, Accuracy, and Fluency (CAF)--the three dimensions in evaluating L2 speakers' oral proficiency, and the subsequent impact indices…

Background: Competency models provide useful frameworks for organizing learning and assessment programs, but their construction is both time intensive and subject to perceptual biases. Some aspects of model development may be particularly well-suited to automation, specifically natural language processing (NLP), which could also help make them…

Emotions are a core semantic component of human communication. Since counsellors are humans we assume that their own state of emotions could affect their intuitional effort when taking decisions concerning their clients. Therefore, the accuracy of detected emotions by counsellors could be doubtful. And this highlights the need for complementing…

Development of reliable rubrics for educational intervention studies that address reading and writing skills is labor-intensive, and could benefit from an automated approach. We compare a main ideas rubric used in a successful writing intervention study to a highly reliable wise-crowd content assessment method developed to evaluate…

A significant barrier to effective communication in a second language is the awareness and accurate reproduction of phonetic sounds absent in the mother tongue. This study investigated whether the automated evaluation of phonetic accuracy using speech recognition technology could improve the pronunciation skills of 105 (88 female, 17 male) Thai…

This study implemented a data-driven approach to identify Chinese high school students' common errors in a Java-based introductory programming course using the data in an automated assessment tool called the Mulberry. Students' error-related behaviors were also analyzed, and their relationships to success in introductory programming were…

The "e-rater"® automated scoring engine used at Educational Testing Service (ETS) scores the writing quality of essays. In the current practice, e-rater scores are generated via a multiple linear regression (MLR) model as a linear combination of various features evaluated for each essay and human scores as the outcome variable. This…