In many large-scale testing programs, examinees are frequently categorized into different performance levels. These classifications are then used to make high-stakes decisions about examinees in contexts such as in licensure, certification, and educational assessments. Numerous approaches to estimating the consistency and accuracy of this…

We have maintained the customer grade system that is being implemented to customers with excellent performance through customer segmentation for years. Currently, financial institutions that operate the customer grade system provide similar services based on the score calculation criteria, but the score calculation criteria vary from the financial…

In massive open online course (MOOC) environments, computer-based analysis of course reviews enables instructors and course designers to develop intervention strategies and improve instruction to support learners' learning. This study aimed to automatically and effectively identify learners' concerned topics within their written reviews. First, we…

Decision consistency (DC) is the reliability of a classification decision based on a test score. In professional credentialing, the decision is often a high-stakes pass/fail decision. The current methods for estimating DC are computationally complex. The purpose of this research is to provide a computationally and conceptually simple method for…

At present, the widespread use of online education platforms has attracted the attention of more and more people. The application of AI technology in online education platform makes multidimensional evaluation of students' ability become the trend of intelligent education in the future. Currently, most existing studies are based on traditional…

The current study aims to evaluate the performance of three non-IRT procedures (i.e., normal approximation, Livingston-Lewis, and compound multinomial) for estimating classification indices when the observed score distribution shows atypical patterns: (a) bimodality, (b) structural (i.e., systematic) bumpiness, or (c) structural zeros (i.e., no…

In this paper, a quantitative system of undergraduates' creative ability is proposed through analyzing characteristics of Amabile creative theory, and the objectivity and feasibility of CAT, TTCT and AMS in creative ability quantification. The academic test scores, TTCT scores and AMS scores are used as the quantitative index of professional…

Here, we describe the development and validation of an automatic assessment system that examines students' hand-drawn visual representations in free-response items. The data were collected from 1,028 students in the second through 11th grades in South Korea using two items from the Test About Particles in a Gas questionnaire (Novick &…

When item scores are ordered categorical, categorical omega can be computed based on the parameter estimates from a factor analysis model using frequentist estimators such as diagonally weighted least squares. When the sample size is relatively small and thresholds are different across items, using diagonally weighted least squares can yield a…

Purpose: This study aims to analyse the insertion of Sustainability in Management undergraduate courses in Brazilian higher education institutions (HEIs). Design/methodology/approach: Through content analysis, the authors have mapped the curricular grids of the Management courses simultaneously best classified in an independent national ranking…

More than 40 percent of students in California's public schools speak a language other than English at home. In the 2016-17 school year, 21 percent of all students, or more than 1.3 million, were English Learners (ELs). When former English Learners are included, the population of "ever ELs" expands to 38 percent of all K-12 students in…

This report describes research carried out over two distinct reclassification eras between the 2002-03 and 2013-14 academic years in the Los Angeles Unified (LAUSD) and San Diego Unified School Districts (SDUSD). Using student-level data and the multiple reclassification policies, the researchers are able to discern whether reclassification takes…

Research in the field of Education has produced a corpus of studies dealing with the specific relationship between school building condition and student achievement reporting positive results. Yet, studies have been completed reporting no significant difference in achievement scores from students in buildings in poor and good condition. The…

Introduction: The use of averaged topic-level scores can result in the loss of valuable data and can cause misinterpretation of the effectiveness of system performance. This study aims to use the scores of each document to evaluate document retrieval systems in a pairwise system evaluation. Method: The chosen evaluation metrics are document-level…

This document includes two technical appendices that accompany the main report, "English Learner Trajectories and Reclassification." The two appendices include: (1) Methodology; and (2) Supporting Tables and Figures. [For the full report, see ED603764.]