Tatsuoka (1984) suggested several extended caution indices and their standardized versions that have been used as person-fit statistics by researchers such as Drasgow, Levine, and McLaughlin (1987), Glas and Meijer (2003), and Molenaar and Hoijtink (1990). However, these indices are only defined for tests with dichotomous items. This paper extends…

In this article, the authors applied profile analysis to Maths exam data to demonstrate how different exam forms, differing in difficulty and length, can be reported and easily interpreted. The results were presented for different groups of participants and for different institutions in different Maths domains by evaluating the balance. Some…

The growing use of scales in survey questionnaires warrants the need to address how does polytomous differential item functioning (DIF) affect observed scale score comparisons. The aim of this study is to investigate the impact of DIF on the type I error and effect size of the independent samples t-test on the observed total scale scores. A…

Results from Rasch analysis of GCSE and GCE A level data over a period of four years suggest that the standards of examinations in different subjects are not consistent in terms of the levels of the latent trait specified in the Rasch model required to achieve the same grades. Variability in statistical standards between subjects exists at both…

The present study aimed to represent the innovative assessments that support students' learning in STEM education through using the integrative framework for Cognitive Diagnostic Modeling (CDM). This framework is based on three components, cognition, observation, and interpretation (National Research Council, 2001). Specifically, this dissertation…

Variation in test performance among examinees from different regions or national jurisdictions is often partially attributed to differences in the degree of content correspondence between local school or training program curricula, and the test of interest. This posited relationship between test-curriculum correspondence, or "alignment,"…

This module prepares the reader to use statistical procedures to detect differentially functioning test items. The Mantel-Haenszel statistic, logistic regression, the SIBTEST procedure, the Standardization procedure, and various item response theory-based procedures are presented. Theoretical frameworks, strengths and weaknesses, and…

The interaction of subjects with test items is investigated. It is suggested that psychometricians rely too heavily on inferences about the nature of the interaction. An approach to data analysis is proposed which is more directly related to this interaction. (JKS)

The restrictions on item difficulties that must be met when binomial models are applied to domain-referenced testing are examined. Both a deterministic and a stochastic conception of item responses are discussed with respect to difficulty and Guttman-type items. (Author/BH)

Several special cases of the general Rasch model, varying in complexity, were investigated to determine whether they could successfully model realistic multidimensional item response data. Whether the parameters of the model could be readily interpreted was also investigated. The models investigated included: (1) the vector model; (2) the product…

Alternate versions of Hutchinson's theory were compared, and one which implies the existence of partial knowledge was found to be better than one which implies that an appropriate measure of ability is obtained by applying the conventional correction for guessing. (Author/PN)

The Research Services of the National Education Association (NEA) conducted a nationwide teacher opinion poll (TOP) based upon a stratified disproportionate two-state cluster sample of classroom teachers. This research study was conducted to test the hypothesis that the order of presentation of items would make no difference in the conclusions…

An ideal process is outlined for test item development and the study of item responses to ensure that tests are sound. Qualitative and quantitative methods are used to assess the item-level validity evidence for high-stakes examinations. A checklist for assessment is provided. (SLD)

An approach is described that reveals the hierarchical test structure (HTS) based on the cognitive demands of the test items, and conducts a linear trait modeling by using the HST elements as item difficulty components. This approach, referred to as the Hierarchical Latent Trait Approach (HLTA), employs an algorithm that allows all test items to…

Although item bias statistics are widely recommended for use in test development and test analysis work, problems arise in their interpretation. The purpose of the present research was to evaluate the validity of logistic test models and computer simulation methods for providing a frame of reference for item bias statistic interpretations.…