Integrative data analyses have recently been shown to be an effective tool for researchers interested in synthesizing datasets from multiple studies in order to draw statistical or substantive conclusions. The actual process of integrating the different datasets depends on the availability of some common measures or items reflecting the same…

This article presents a general-purpose method for detecting cheating in online courses, which combines anomaly detection and supervised machine learning. Using features that are rooted in psychometrics and learning analytics literature, and capture anomalies in learner behavior and response patterns, we demonstrate that a classifier that is…

Data harmonization is an emerging approach to strategically combining data from multiple independent studies, enabling addressing new research questions that are not answerable by a single contributing study. A fundamental psychometric challenge for data harmonization is to create commensurate measures for the constructs of interest across…

The purpose of this study is to illustrate the use of functional data analysis (FDA) as a general methodology for analyzing person response functions (PRFs). Applications of FDA to psychometrics have included the estimation of item response functions and latent distributions, as well as differential item functioning. Although FDA has been…

Composite scores are an exceptionally important psychometric tool for behavioral science research applications. A prototypical example occurs with self-report data, where researchers routinely use questionnaires with multiple items that tap into different features of a target construct. Item-level missing data are endemic to composite score…

Computer-based and web-based testing have become increasingly popular in recent years. Their popularity has dramatically expanded the availability of response time data. Compared to the conventional item response data that are often dichotomous or polytomous, response time has the advantage of being continuous and can be collected in an…

The fit of an item response model is typically conceptualized as whether a given model could have generated the data. We advocate for an alternative view of fit, "predictive fit", based on the model's ability to predict new data. We derive two predictive fit metrics for item response models that assess how well an estimated item response…

Conducting valid and reliable empirical research in the prevention sciences is an inherently difficult and challenging task. Chief among these is the need to obtain numerical scores of underlying theoretical constructs for use in subsequent analysis. This challenge is further exacerbated by the increasingly common need to consider multiple…

Effective learning outcome modeling is crucial to the success of learning evaluation in education. In the digital age, the movement towards online learning and computerized assessments has resulted in an explosion of structured and unstructured educational data (e.g., learners' problem-solving process data), which offers new opportunities for…

Categorical data become increasingly ubiquitous in the modern big data era. In this dissertation, we propose novel statistical learning and inference methods for large-scale categorical data, focusing on latent variable models and their applications to psychometrics. In psychometric assessments, the subjects' underlying aptitude often cannot be…

We propose a novel model, Wide & Deep Item Response Theory (Wide & Deep IRT), to predict the correctness of students' responses to questions using historical clickstream data. This model combines the strengths of conventional Item Response Theory (IRT) models and Wide & Deep Learning for Recommender Systems. By leveraging clickstream…

The "2nd Annual WPI-UMASS-UPENN EDM Data Mining Challenge" required contestants to predict efficient testtaking based on log data. In this paper, we describe our theory-driven and psychometric modeling approach. For feature engineering, we employed the Log-Normal Response Time Model for estimating latent person speed, and the Generalized…

Reasoning for complexity is a fundamental competency in these complex times for solutions to social problems and decision-making. The purpose of this paper is to demonstrate the validity and reliability of the eComplexity instrument by presenting its psychometric properties. The instrument consists of a Likert-type scale questionnaire designed to…

Many emotion regulation assessments have been developed for research purposes, but few are frequently used in schools despite the rapid growth of social and emotional learning programs with an explicit focus on emotion regulation in schools. This systematic review provides an overview of emotion regulation assessments that have been utilized with…

Promoting understanding of the epistemologies of science has long been the primary objective in science education, and can be viewed as a form of science learning outcome. Many studies have attempted to understand learners' conceptions of epistemology in science from various perspectives and methods; however, no recent reviews have focused on the…