The major purpose of this study is to examine the quality of ratings assigned to CR (constructed-response) questions in large-scale assessments from the perspective of Rasch Measurement Theory. Rasch Measurement Theory provides a framework for the examination of rating scale category structure that can yield useful information for interpreting the…

This dissertation proposes two modified cognitive diagnostic models (CDMs), the deterministic, inputs, noisy, "and" gate with hierarchy (DINA-H) model and the deterministic, inputs, noisy, "or" gate with hierarchy (DINO-H) model. Both models incorporate the hierarchical structures of the cognitive skills in the model estimation…

Validly assessing students with disabilities has been a challenge for decades but is increasingly vital to educational policy and practice in the current era of accountability. Numerous technological and policy developments have occurred in the past several years with the emergence and decline of various forms of alternate assessments. This study…

There have been great strides made in research on vocabulary in the last 30 years. However, there has been relatively little progress in the development of new vocabulary tests. This may be due in some degree to the impressive contributions made by tests such as the Vocabulary Levels Test (Nation, 1983; Schmitt et al., 2001) and the Word…

The purpose of this article was to describe an initial step developing a new scale to identify individuals with learning disabilities (LD) and test anxiety. Eighty-eight students answered the "Minnesota Multiphasic Personality Inventory-2" (MMPI-2). The participants were drawn from the following three groups: (a) adults with LD and test…

In this dissertation, I explore unsupervised techniques for the task of automatic short answer grading. I compare a number of knowledge-based and corpus-based measures of text similarity, evaluate the effect of domain and size on the corpus-based measures, and also introduce a novel technique to improve the performance of the system by integrating…

A new spatial ability test was administered online to 223 undergraduate students enrolled in introductory science courses. The 30-item multiple choice test measures individual differences in ability to identify the two-dimensional cross section of a three-dimensional geometric solid, a skill that has been identified as important in science,…

In this study we compared five item selection procedures using three ability estimation methods in the context of a mixed-format adaptive test based on the generalized partial credit model. The item selection procedures used were maximum posterior weighted information, maximum expected information, maximum posterior weighted Kullback-Leibler…

Background: Although McNemar Test is the most appropriate tool for analyzing pre-post differences in dichotomous items (e.g., "yes" or "no", "correct" or "incorrect", etc.), many scholars have noted the inappropriate use of Pearson's Chi-square Test by researchers, including social scientists and evaluators,…

It is shown that deviations of estimated from true values of item difficulty parameters, caused for example by item calibration errors, the neglect of randomness of item difficulty parameters, testlet effects, or rule-based item generation, can lead to systematic bias in point estimation of person parameters in the context of adaptive testing.…

Multiple-indicators multiple-causes (MIMIC) modeling is often used to test a latent group mean difference while assuming the equivalence of factor loadings and intercepts over groups. However, this study demonstrated that MIMIC was insensitive to the presence of factor loading noninvariance, which implies that factor loading invariance should be…

Andersen (1995, 2002) proves a theorem relating variances of parameter estimates from samples and subsamples and shows its use as an adjunct to standard statistical analyses. The authors show an application where the theorem is central to the hypothesis tested, namely, whether random guessing to multiple choice items affects their estimates in the…

This study investigated item exposure control procedures under various combinations of item pool characteristics and ability distributions in computerized adaptive testing based on the partial credit model. Three variables were manipulated: item pool characteristics (120 items for each of easy, medium, and hard item pools), two ability…

In this article we propose a model-free diagnostic for single-peakedness (unimodality) of item responses. Presuming a unidimensional unfolding scale and a given item ordering, we approximate item response functions of all items based on ordered conditional means (OCM). The proposed OCM methodology is based on Thurstone & Chave's (1929) "criterion…

This study assessed whether the Center for Epidemiological Studies Depression Scale (CES-D) functions equivalently in assessing depressive symptom severity in lesbian, bisexual, and heterosexual women. Using differential item functioning methods, the authors examined (a) whether there is a bias in CES-D total scores and in individual item scores…