This study utilizes an analysis of diminishing returns to examine the relationship between sample size and item parameter estimation precision when utilizing the Masters' Partial Credit Model for polytomous items. Item data from the standardization of the Batelle Developmental Inventory, 3rd Edition were used. Each item was scored with a…

A procedure was developed to analyze data with missing observations by extracting data from a sparsely filled data matrix into analyzable smaller subsets of data. This subdividing method, based on the conceptual framework of meta-analysis, was accomplished by creating data sets that exhibit structural designs and then pooling variance components…

An evaluation of the variation of item estimates was conducted for the multidimensional extension of the logistic item response theory (MIRT) model. The empirically determined standard errors (SEs) of marginal maximum likelihood estimation (MMLE)/Bayesian item estimates from 40 items from the ACT Assessment (Form 24b, 1985) were obtained when the…

The Job Satisfaction Survey (JSS) (P. Spector, 1985 and 1992) is a 36-item survey instrument designed to measure 9 aspects of job satisfaction, including: (1) pay; (2) promotion; (3) supervision; (4) benefits; (5) contingent rewards; (6) operating procedures; (7) co-workers; (8) nature of work; and (9) communication. In addition to measuring the…

A method is presented for the simultaneous analysis of differential item functioning (DIF) in multi-factor situations. The method is unique in that it combines item response theory (IRT) and analysis of variance (ANOVA), takes a simultaneous approach to multifactor DIF analysis, and is capable of capturing interaction and controlling for possible…

This study explores the relative merits of a potentially useful item response theory (IRT) linking design: using a single set of anchor items with fixed common item parameters (FCIP) during the calibration process. An empirical study was conducted to investigate the appropriateness of this linking design using 6 groups of students taking 6 forms…

Potential benefits of using item response theory in test construction are evaluated, based on the experience and evidence accumulated during 9 years of using a three-parameter model in the construction of major achievement batteries. Specific benefits covered include obtaining sample-free item calibrations and item-free person measurement,…

The setting of standards involves subjective value judgments. The inherent arbitrariness of specific standards has been severely criticized by Glass. His antagonists agree that standard setting is a judgmental task but they have pointed out that arbitrariness in the positive sense of serious judgmental decisions is unavoidable. Further, small…

In most reliability studies, the precision of a reliability estimate varies inversely with the number of examinees (sample size). Thus, to achieve a given level of accuracy, some minimum sample size is required. An approximation for this minimum size may be made if some reasonable assumptions regarding the mean and standard deviation of the test…

Previous studies have shown that, when administered a self-adapted test, a few examinees will choose item difficulty levels that are not well-matched to their proficiencies, resulting in high standard errors of proficiency estimation. This study investigated whether the previously observed effects of a self-adapted test--lower anxiety and higher…

In order to determine the effectiveness of multidimensional scaling (MDS) in recovering the dimensionality of a set of dichotomously-scored items, data were simulated in one, two, and three dimensions for a variety of correlations with the underlying latent trait. Similarity matrices were constructed from these data using three margin-sensitive…

This paper deals with test fairness regarding a test consisting of two parts: (1) a "common" section, taken by all students; and (2) a "variable" section, in which some students may answer a different set of questions from other students. For example, a test taken by several thousand students each year contains a common multiple-choice portion and…

A primary goal of this study was to determine the extent to which item difficulty was related to item position and, if a significant relationship was found, to suggest adjustments to predicted item difficulty that reflect differences in item position. Item response data from the Medical College Admission Test (MCAT) were analyzed. A data set was…

Mail questionnaire items that are susceptible to order effects were examined using data from 168 questionnaires in a Florida Cooperative Extension Service evaluation. Order effects were found for multiple-response and attributive questions but not for single-response items. Order also interacted with question complexity, social desirability, and…

Measurement disturbances, such as guessing, startup, and plodding, often result in an examinee's ability being either over- or under-estimated by the maximum likelihood estimation employed in latent trait psychometric models. Several authors have suggested methods to lessen the impact of unexpected responses on the ability estimation process. This…