We determined the recommendations for preparing and scoring constructed-response (CR) test items in 25 sources (textbooks and chapters) on educational and psychological measurement. The project was similar to Haladyna's (2004) analysis for multiple-choice items. We identified 12 recommendations for preparing CR items given by multiple sources,…

This article investigates several methods of identifying individual guessers from their response data. Both the posterior probability method and the likelihood ratio method are based on the two-state mixture modeling approach to response times. The accuracy method is based on response accuracy data. Results from the simulation study showed that…

False recognition, broadly defined as a claim to remember something that was not encountered previously, can arise for multiple reasons. For instance, a distinction can be made between conceptual false recognition (i.e., false alarms resulting from semantic or associative similarities between studied and tested items) and perceptual false…

The researchers of this study investigated how frequently a set of 17 non-academic behavioral indicators were used to determine impairment of master's-level counseling students that resulted in remediation and termination. Thirty-five academic unit leaders of the Council for Accreditation of Counseling and Related Educational Programs (CACREP)…

This research project addresses the validity of assessments used to measure the performance of English language learners (ELLs), such as those mandated by the No Child Left Behind Act of 2001 (NCLB, 2002). The goals of the research are to help educators understand and improve ELL performance by investigating the validity of their current…

Despite favorable psychometric properties, the Generalized Anxiety Disorder Questionnaire for the "Diagnostic and Statistical Manual of Mental Disorders" (4th ed.) (GAD-Q-IV) does not have a known factor structure, which calls into question use of its original weighted scoring system (usually referred to as the dimensional score).…

This paper describes the development and validation of the Cognate Awareness Test (CAT), which measures cognate awareness in Spanish-speaking English Language Learners (ELLs) in fourth and fifth grade. An investigation of differential performance on the two subtests of the CAT (cognates and noncognates) provides evidence that the instrument is…

In the context of paper and pencil testing, the frequency of the exposure of items is usually controlled through policies that regulate both the reuse of test forms and the frequency with which a candidate may retake the test. In the context of computerized adaptive testing, where item pools are large and expensive to produce and testing can be on…

In this paper, formulas for computing the weights that maximize the reliability of a test with multiple parts are derived using a congeneric model. A direct derivation for the three-part test and case and a two-step derivation for the n-part case are presented, and results for these two approaches are shown to be consistent for the three-part…

The decision to use a forced-choice test item format versus an item format where choice is not forced (e.g., a Likert scale) might best be determined by the nature of the information sought since the difficult decisions required for forced-choice formats may result in a different scaling than an unforced method. If a forced choice is desired,…

A simple technique, developed by A. Phillips (1987) is used to approximate the covariance between the Mantel-Haenszel log-odds-ratio estimator for a 2 x 2 x k table and the sample marginal proportions. These results are then applied to obtain an approximate variance estimate of an adjusted risk difference based on the Mantel-Haenszel odds-ratio…

Three aspects of the usual approach to assessing local item dependency, Yen's "Q" (H. Huynh, H. Michaels, and S. Ferrara, 1995), deserve further investigation. Pearson correlation coefficients do not distribute normally when the coefficients are large, and thus cannot quantify the dependency well. In the second place, the accuracy of…

Methods for detecting item score patterns that are unlikely (aberrant) given that a parametric item response theory (IRT) model gives an adequate description of the data or given the responses of the other persons in the group are discussed. The emphasis here is on the latter group of statistics. These statistics can be applied when a…

Computer simulations under three conditions of polytomous differential item functioning (DIF) compared the ability of three different statistical procedures to detect nonuniform DIF. The procedures were a nominal and an ordinal extension of the Mantel-Haenszel statistic, and logistic discriminant function analysis. Results showed that only the…

This paper addresses questions of whether positively- and negatively-worded items measure the same construct and whether the rating scale categories "strongly agree" to "strongly disagree" are used in the same way for both types of items. Item response theory (IRT), specifically the Andrich Rating Scale Model (B. Wright and G.…