Background: Although McNemar Test is the most appropriate tool for analyzing pre-post differences in dichotomous items (e.g., "yes" or "no", "correct" or "incorrect", etc.), many scholars have noted the inappropriate use of Pearson's Chi-square Test by researchers, including social scientists and evaluators,…

It is shown that deviations of estimated from true values of item difficulty parameters, caused for example by item calibration errors, the neglect of randomness of item difficulty parameters, testlet effects, or rule-based item generation, can lead to systematic bias in point estimation of person parameters in the context of adaptive testing.…

Multiple-indicators multiple-causes (MIMIC) modeling is often used to test a latent group mean difference while assuming the equivalence of factor loadings and intercepts over groups. However, this study demonstrated that MIMIC was insensitive to the presence of factor loading noninvariance, which implies that factor loading invariance should be…

Andersen (1995, 2002) proves a theorem relating variances of parameter estimates from samples and subsamples and shows its use as an adjunct to standard statistical analyses. The authors show an application where the theorem is central to the hypothesis tested, namely, whether random guessing to multiple choice items affects their estimates in the…

This study investigated item exposure control procedures under various combinations of item pool characteristics and ability distributions in computerized adaptive testing based on the partial credit model. Three variables were manipulated: item pool characteristics (120 items for each of easy, medium, and hard item pools), two ability…

In this article we propose a model-free diagnostic for single-peakedness (unimodality) of item responses. Presuming a unidimensional unfolding scale and a given item ordering, we approximate item response functions of all items based on ordered conditional means (OCM). The proposed OCM methodology is based on Thurstone & Chave's (1929) "criterion…

This study assessed whether the Center for Epidemiological Studies Depression Scale (CES-D) functions equivalently in assessing depressive symptom severity in lesbian, bisexual, and heterosexual women. Using differential item functioning methods, the authors examined (a) whether there is a bias in CES-D total scores and in individual item scores…

Four maltreatment items were examined from Wave III (N = 13,516) of the National Longitudinal Study of Adolescent Health. Item analysis, confirmatory factor analysis, cross-validation, reliability estimates, and convergent validity coefficients strongly supported the validity of using the four items as a unidimensional composite. Implications for…

We investigated examinees' cognitive processes while they solved selected items from the Multistate Bar Exam (MBE), a high-stakes professional certification examination. We focused on ascertaining those mental processes most frequently used by examinees, and the most common types of errors in their thinking. We compared the relationships between…

In a recent article, Nadelson and Southerland (2010. Development and preliminary evaluation of the Measure of Understanding of Macroevolution: Introducing the MUM. "The Journal of Experimental Education", 78, 151-190) reported on their development of a multiple-choice concept inventory intended to assess college students' understanding…

This study explored how students viewed the role of ideas for knowledge work and how such a view was related to their inquiry activities. Data mainly came from students' online interaction logs, group discussion and inquiry, and a survey concerning the role of ideas for knowledge work. The findings suggest that knowledge building was conducive to…

Many psychological scales written in the Likert format include reverse worded (RW) items in order to control acquiescence bias. However, studies have shown that RW items often contaminate the factor structure of the scale by creating one or more method factors. The present study examines an alternative scale format, called the Expanded format,…

Dynamic Assessment (DA) is a postmodern notion in testing which sees instruction and assessment as inextricably mingled contending that learners will progress if provided with dynamic interactions. The main purpose of the study is to see if the scores generated by the computerized dynamic assessment (C-DA) which is grounded in Vygotsky's…

High-stakes testing is used for the purposes of providing results that have important consequences. Validity is the cornerstone upon which all measurement systems are built. This study applied the Item Response Theory principles to analyse Northwest University Kano Post-UTME Economics test items. The developed fifty (50) economics test items was…

As part of an international effort to develop theory and best practices for teaching languages, the U.S. military has, since the American Revolution, been a leading supporter of literacy education to improve the job performance of soldiers. One important aspect of literacy education today--which continues to be a priority for government agencies,…