In many large-scale testing programs, examinees are frequently categorized into different performance levels. These classifications are then used to make high-stakes decisions about examinees in contexts such as in licensure, certification, and educational assessments. Numerous approaches to estimating the consistency and accuracy of this…

Consistency and coverage are two core parameters of model fit used by configurational comparative methods (CCMs) of causal inference. Among causal models that perform equally well in other respects (e.g., robustness or compliance with background theories), those with higher consistency and coverage are typically considered preferable. Finding the…

High dropout rate exists universally in massive open online courses (MOOCs) due to the separation of teachers and learners in space and time. Dropout prediction using the machine learning method is an extremely important prerequisite to identify potential at-risk learners to improve learning. It has attracted much attention and there have emerged…

In recent years, the application of explainability techniques to automated essay scoring and automated short-answer grading (ASAG) models, particularly those based on transformer architectures, has gained significant attention. However, the reliability and consistency of these techniques remain underexplored. This study systematically investigates…

In this study, we estimated the relationship between two popular measures of teacher effectiveness--teachers' value-added model (VAM) estimates, represented in this study via median growth percentiles (MGPs), and teachers' observational scores, derived from the TAP System for Teacher and Student Advancement. We examined the relationship between…

Thurstonian forced-choice modeling is considered to be a powerful new tool to estimate item and person parameters while simultaneously testing the model fit. This assessment approach is associated with the aim of reducing faking and other response tendencies that plague traditional self-report trait assessments. As a result of major recent…

We have maintained the customer grade system that is being implemented to customers with excellent performance through customer segmentation for years. Currently, financial institutions that operate the customer grade system provide similar services based on the score calculation criteria, but the score calculation criteria vary from the financial…

Decision consistency (DC) is the reliability of a classification decision based on a test score. In professional credentialing, the decision is often a high-stakes pass/fail decision. The current methods for estimating DC are computationally complex. The purpose of this research is to provide a computationally and conceptually simple method for…

This study investigates the evolving landscape of green universities by analyzing and comparing rankings from 2018 to 2022. It expands beyond the single score offered by the UI GreenMetric, employing Multi-Criteria Decision-Making (MCDM) techniques to evaluate universities from diverse perspectives. Focusing on the top 50 universities from 2022,…

Purpose: The purpose of this study is to determine how organizational characteristics at higher education institutions (HEI) influence their sustainability performance as measured by the advancement of sustainability in higher education's sustainability tracking, assessment and rating system (STARS). Design/methodology/approach: This analysis…

Comparative Judgement (CJ) is an evaluation method, typically conducted online, whereby a rank order is constructed, and scores calculated, from judges' pairwise comparisons of performances. CJ has been researched in various educational contexts, though only rarely in English as a Foreign Language (EFL) writing settings, and is generally agreed to…

The main objective of this Methods Showcase Article is to show how the technique of adaptive comparative judgment, coupled with a crowdsourcing approach, can offer practical solutions to reliability issues as well as to address the time and cost difficulties associated with a text-based approach to proficiency assessment in L2 research. We…

Clients display considerable variations in functioning across the contexts that encompass their social environments (e.g., home, school/workplace, peer interactions). No single measurement method can fully capture these variations. Yet, assessors must balance the need to accurately capture clients' clinical presentations, and at the same time…

Fuzzy sets have experienced multiple expansions since their conception to enhance their capacity to convey complex information. Intuitionistic fuzzy sets, image fuzzy sets, q-rung orthopair fuzzy sets, and neutrosophic sets are a few of these extensions. Researchers and academics have acquired a lot of information about their theories and methods…

Many large-scale performance assessments include score resolution procedures for resolving discrepancies in rater judgments. The goal of score resolution is conceptually similar to person fit analyses: To identify students for whom observed scores may not accurately reflect their achievement. Previously, researchers have observed that…