Well-crafted rubrics create a shared language that lets teachers and students work together. Rubrics are typically used to judge the level of students' understanding and skills or quality of a product. High-quality rubrics can also give students and teachers feedback to improve teaching and learning. Authors define "effective" feedback…

We systematically compared two coding approaches to generate training datasets for machine learning (ML): (1) a holistic approach based on learning progression levels; and (2) a dichotomous, analytic approach of multiple concepts in student reasoning, deconstructed from holistic rubrics. We evaluated four constructed response assessment items for…

Effort-moderated (E-M) scoring is intended to estimate how well a disengaged test taker would have performed had they been fully engaged. It accomplishes this adjustment by excluding disengaged responses from scoring and estimating performance from the remaining responses. The scoring method, however, assumes that the remaining responses are not…

This article explores the application of journal quality and credibility evaluation tools to library science publications. The researchers investigate quality and credibility attributes of forty-eight peer-reviewed library science journals with open access components using two evaluative tools developed and published by librarians. The results…

Recent developments in computerized scoring via semantic distance have provided automated assessments of verbal creativity. Here, we extend past work, applying computational linguistic approaches to characterize salient features of creative text. We hypothesize that, in addition to semantic diversity, the degree to which a story includes…

Assessing student knowledge based on their writing using traditional qualitative methods is time-consuming. To improve speed and consistency of text analysis, we present our mixed methods development of a machine learning predictive model to analyze student writing. Our approach involves two stages: first an exploratory sequential design, and…

Understanding student cognitive states is essential for assessing human learning. The deep neural networks (DNN)-inspired cognitive state prediction method improved prediction performance significantly; however, the lack of explainability with DNNs and the unitary scoring approach fail to reveal the factors influencing human learning. Identifying…

For randomized trials that use text as an outcome, traditional approaches for assessing treatment impact require that each document first be manually coded for constructs of interest by trained human raters. This process, the current standard, is both time-consuming and limiting: even the largest human coding efforts are typically constrained to…

This study investigated the impact of scoring polytomous items later on measurement precision, classification accuracy, and test security in mixed-format adaptive testing. Utilizing the shadow test approach, a simulation study was conducted across various test designs, lengths, number and location of polytomous item. Results showed that while…

Background: Complex skills, like analytical thinking, are essential in preparing students for future professions. Serious games hold potential to stimulate the online acquisition of such professional skills in an active and experiential way. Objective: Rubrics are proven assessment and evaluation instruments, but were never directly integrated…

The study aimed to explore the perception of pre-service teachers regarding the use of the red pen for corrections. A non-random sample of twelve (12) respondents, consisting of three experienced tutors from the Mathematics, Science, and English Departments, along with nine pre-service teachers, were interviewed. The data was analysed…

When analyzing treatment effects on test scores, researchers face many choices and competing guidance for scoring tests and modeling results. This study examines the impact of scoring choices through simulation and an empirical application. Results show that estimates from multiple methods applied to the same data will vary because two-step models…

The purpose of this mixed methods study was to determine how communication evaluation by clinical faculty compared in a clinical setting versus a simulation setting for Bachelor of Science in Nursing (BSN) students. Fifty (50) BSN students from three different schools of nursing were scored using the Interprofessional Situation, Background,…

The purpose of this qualitative content analysis study is to investigate the themes and potential gaps in quality assurance rubrics and scorecards used to evaluate online courses in higher education. For the utility of this study, the themes and potential gaps in quality assurance rubrics and scorecards is generally defined as the recurring…

Automated Essay Scoring (AES) is a well-studied problem in Natural Language Processing applied in education. Solutions vary from handcrafted linguistic features to large Transformer-based models, implying a significant effort in feature extraction and model implementation. We introduce a novel Automated Machine Learning (AutoML) pipeline…