When scores are used to make decisions about respondents, it is of interest to estimate classification accuracy (CA), the probability of making a correct decision, and classification consistency (CC), the probability of making the same decision across two parallel administrations of the measure. Model-based estimates of CA and CC computed from the…

Over the past decade, statisticians have developed methods to improve generalizations from nonrandom samples using propensity score methods. While these methods contribute to generalization research, their effectiveness is limited by small sample sizes. Small area estimation is a class of model-based methods that address the imprecision due to…

The causal view of categories assumes that categories are represented by features and their causal relations. To study the effect of causal knowledge on categorization, researchers have used Bayesian causal models. Within that framework, categorization may be viewed as dependent on a likelihood computation (i.e., the likelihood of an exemplar with…

Simulation studies involving mixture models inevitably aggregate parameter estimates and other output across numerous replications. A primary issue that arises in these methodological investigations is label switching. The current study compares several label switching corrections that are commonly used when dealing with mixture models. A growth…

Purpose: Course evaluations are formative and are used to evaluate learnings of the students for a course. Anomalies in the evaluation process can lead to a faulty educational outcome. Learning analytics and educational data mining provide a set of techniques that can be conveniently applied to extensive data collected as part of the evaluation…

In a range of settings, human operators make decisions with the assistance of automation, the reliability of which can vary depending upon context. Currently, the processes by which humans track the level of reliability of automation are unclear. In the current study, we test cognitive models of learning that could potentially explain how humans…

Though individual categorization or decision processes have been studied separately in many previous investigations, few studies have investigated how they interact by using a two-stage task of first categorizing and then deciding. To address this issue, we investigated a categorization-decision task in two experiments. In both, participants were…

Place-based initiatives often use resident surveys to inform and evaluate interventions. Sampling based on well-defined sampling frames is important but challenging for initiatives that target subpopulations. Databases that enumerate total population counts can produce overinclusive sampling frames, resulting in costly outreach to ineligible…

Testing programs are confronted with the decision of whether to report individual scores for examinees that have engaged in rapid guessing (RG). As noted by the "Standards for Educational and Psychological Testing," this decision should be based on a documented criterion that determines score exclusion. To this end, a number of heuristic…

We tested 6- and 8-month-old White and non-White infants (N = 53 total, 28 girls) from Northern California in a visual search task to determine whether a unique item in an otherwise homogeneous display (a singleton) attracts attention because it is a unique singleton and "pops out" in a categorical manner, or whether attention instead…

One common score reported from diagnostic classification assessments is the vector of posterior means of the skill mastery indicators. As with any assessment, it is important to derive and report estimates of the reliability of the reported scores. After reviewing a reliability measure suggested by Templin and Bradshaw, this article suggests three…

When text datasets are very large, manually coding line by line becomes impractical. As a result, researchers sometimes try to use machine learning algorithms to automatically code text data. One of the most popular algorithms is topic modeling. For a given text dataset, a topic model provides probability distributions of words for a set of…

This study used latent class analysis (LCA) with binary observed indicators to identify latent classes of victimization, based on the extent to which adolescents in the U.S. experienced traditional victimization and cyber-victimization. Data were collected by the National Center for Education Statistics and the Bureau of Justice Statistics using…

The U.S. Department of Education's Office of Special Education Programs collects information from all states on the education of students with disabilities, including the number of students that exit special education and transfer back to general education. Yet, little is known about students that are declassified. In this study, we used data from…

In educational and psychological research, we are often interested in discrete latent states of individuals responding to an assessment (e.g., proficiency or non-proficiency on educational standards, the presence or absence of a psychological disorder). Diagnostic classification models (DCMs; also called cognitive diagnostic models [CDMs]) are a…