To address response style or bias in rating scales, forced-choice items are often used to request that respondents rank their attitudes or preferences among a limited set of options. The rating scales used by raters to render judgments on ratees' performance also contribute to rater bias or errors; consequently, forced-choice items have recently…

Modeling and predicting student learning in computer-based environments often relies solely on sequences of accuracy data. Previous research suggests that it does not only matter what we learn, but also how we learn. The detection and analysis of learning behavior becomes especially important, when dealing with open-ended exploration environments,…

Course selection is a crucial and challenging problem that students have to face while navigating through an undergraduate degree program. The decisions they make shape their future in ways that they cannot conceive in advance. Available departmental sample degree plans are not personalized for each student, and personal discussion time with an…

Metrics including Cohen's kappa, precision, recall, and F[subscript 1] are common measures of performance for models of discrete student states, such as a student's affect or behaviour. This study examined discrete model metrics for previously published student model examples to identify situations where metrics provided differing perspectives on…

Mokken scale analysis (MSA) is a probabilistic-nonparametric approach to item response theory (IRT) that can be used to evaluate fundamental measurement properties with less strict assumptions than parametric IRT models. This instructional module provides an introduction to MSA as a probabilistic-nonparametric framework in which to explore…

To assist new scientists in the transition to independent research careers, the National Institutes of Health (NIH) implemented an Early Stage Investigator (ESI) policy beginning with applications submitted in 2009. During the review process, the ESI designation segregates applications submitted by investigators who are within 10 years of…

Electronic portfolios (E-portfolios) are crucial means for workplace-based assessment and feedback. Although E-portfolios provide a useful approach to view each learner's progress, so far options for personalized feedback and potential data about a learner's performances at the workplace often remain unexploited. This paper advocates that…

Bayesian Knowledge Tracing (BKT) has been in wide use for modeling student skill acquisition in Intelligent Tutoring Systems (ITS). BKT tracks and updates student's latent mastery of a skill as a probability distribution of a binary variable. BKT does so by accounting for observed student successes in applying the skill correctly, where success is…

Parceling is referred to as a procedure for computing sums or average scores across multiple items. Parcels instead of individual items are then used as indicators of latent factors in the structural equation modeling analysis (Bandalos 2002, 2008; Little et al., 2002; Yang, Nay, & Hoyle, 2010). Item parceling may be applied to alleviate some…

As adaptive tutoring systems grow increasingly popular for the completion of classwork and homework, it is crucial to assess the manner in which students are scored within these platforms. The majority of systems, including ASSISTments, return the binary correctness of a student's first attempt at solving each problem. Yet for many teachers,…

Bayesian techniques have become increasingly present in the social sciences, fueled by advances in computer speed and the development of user-friendly software. In this paper, we forward the use of Bayesian Asymmetric Regression (BAR) to monitor intervention responsiveness when using Curriculum-Based Measurement (CBM) to assess oral reading…

Latent variable models, such as the popular Knowledge Tracing method, are often used to enable adaptive tutoring systems to personalize education. However, finding optimal model parameters is usually a difficult non-convex optimization problem when considering latent variable models. Prior work has reported that latent variable models obtained…

In this article, the authors provide a methodological critique of the current standard of value-added modeling forwarded in educational policy contexts as a means of measuring teacher effectiveness. Conventional value-added estimates of teacher quality are attempts to determine to what degree a teacher would theoretically contribute, on average,…

Education systems are increasingly creating rich, longitudinal data sets with frequent, and even real-time, data updates of many student measures, including daily attendance, homework submissions, and exam scores. These data sets provide an opportunity for district and school staff members to move beyond an indicators-based approach and instead…

This article presents the original model of the computer adaptive testing and grade formation, based on scientifically recognized theories. The base of the model is a personalized algorithm for selection of questions depending on the accuracy of the answer to the previous question. The test is divided into three basic levels of difficulty, and the…