Validity evidence based on test content is of essential importance in educational testing. One source for such evidence is an alignment study, which helps evaluate the congruence between tested objectives and those specified in the curriculum. However, the results of an alignment study do not always sufficiently capture the degree to which a test…

The review of existing literature suggests that few researchers have adopted cross-language comparisons to explore how cultural background affects the assessment of reading comprehension of students. In this present study, the researchers independently reviewed and rated all the items of two reading comprehension tests translated from Malay into…

The Reflective Functioning Scale (RFS) was developed to assess individual differences in the ability to mentalize attachment relationships. The RFS assesses mentalization from transcripts of the Adult Attachment Interview (AAI). A global score is given by trained coders on an 11-point scale ranging from antireflective to exceptionally reflective.…

The major purpose of this study is to examine the quality of ratings assigned to CR (constructed-response) questions in large-scale assessments from the perspective of Rasch Measurement Theory. Rasch Measurement Theory provides a framework for the examination of rating scale category structure that can yield useful information for interpreting the…

This dissertation proposes two modified cognitive diagnostic models (CDMs), the deterministic, inputs, noisy, "and" gate with hierarchy (DINA-H) model and the deterministic, inputs, noisy, "or" gate with hierarchy (DINO-H) model. Both models incorporate the hierarchical structures of the cognitive skills in the model estimation…

Validly assessing students with disabilities has been a challenge for decades but is increasingly vital to educational policy and practice in the current era of accountability. Numerous technological and policy developments have occurred in the past several years with the emergence and decline of various forms of alternate assessments. This study…

There have been great strides made in research on vocabulary in the last 30 years. However, there has been relatively little progress in the development of new vocabulary tests. This may be due in some degree to the impressive contributions made by tests such as the Vocabulary Levels Test (Nation, 1983; Schmitt et al., 2001) and the Word…

The purpose of this article was to describe an initial step developing a new scale to identify individuals with learning disabilities (LD) and test anxiety. Eighty-eight students answered the "Minnesota Multiphasic Personality Inventory-2" (MMPI-2). The participants were drawn from the following three groups: (a) adults with LD and test…

In this dissertation, I explore unsupervised techniques for the task of automatic short answer grading. I compare a number of knowledge-based and corpus-based measures of text similarity, evaluate the effect of domain and size on the corpus-based measures, and also introduce a novel technique to improve the performance of the system by integrating…

A new spatial ability test was administered online to 223 undergraduate students enrolled in introductory science courses. The 30-item multiple choice test measures individual differences in ability to identify the two-dimensional cross section of a three-dimensional geometric solid, a skill that has been identified as important in science,…

In this study we compared five item selection procedures using three ability estimation methods in the context of a mixed-format adaptive test based on the generalized partial credit model. The item selection procedures used were maximum posterior weighted information, maximum expected information, maximum posterior weighted Kullback-Leibler…

Background: Although McNemar Test is the most appropriate tool for analyzing pre-post differences in dichotomous items (e.g., "yes" or "no", "correct" or "incorrect", etc.), many scholars have noted the inappropriate use of Pearson's Chi-square Test by researchers, including social scientists and evaluators,…

It is shown that deviations of estimated from true values of item difficulty parameters, caused for example by item calibration errors, the neglect of randomness of item difficulty parameters, testlet effects, or rule-based item generation, can lead to systematic bias in point estimation of person parameters in the context of adaptive testing.…

Multiple-indicators multiple-causes (MIMIC) modeling is often used to test a latent group mean difference while assuming the equivalence of factor loadings and intercepts over groups. However, this study demonstrated that MIMIC was insensitive to the presence of factor loading noninvariance, which implies that factor loading invariance should be…

Andersen (1995, 2002) proves a theorem relating variances of parameter estimates from samples and subsamples and shows its use as an adjunct to standard statistical analyses. The authors show an application where the theorem is central to the hypothesis tested, namely, whether random guessing to multiple choice items affects their estimates in the…