Certain planned-missing designs (e.g., simple-matrix sampling) cause zero covariances between variables not jointly observed, making it impossible to do analyses beyond mean estimations without specialized analyses. We tested a multigroup confirmatory factor analysis (CFA) approach by Cudeck (2000), which obtains a model-estimated…

Examined the effect of complex sampling of items on the measurement of differential item functioning (DIF) using the Mantel-Haenszel procedure through a Monte Carlo study. Suggests the superiority of the pooled booklet method when items are selected for examinees according to a balanced incomplete block design. Discusses implications for other DIF…

This paper points out that there are certain generalizations about directions for guessing and methods of scoring that require that data be derived from random groups design. It supports the viewpoint that it is neither sufficient nor appropriate to make such generalizations on the basis of an analysis of scores obtained from the answer sheets of…

The definition of differential item performance (DIP), often referred to as item bias, is discussed. DIP is suggested as a comprehensive term to encompass item bias (item invalidity which is unfair to certain population subgroups) and instructional bias (a valid reflection of group differences in instruction or background). This study investigated…

A number of examples are presented to illustrate a common flaw in the published research on learning, memory, and instruction. Experimental subjects--often college students--have certain expectations about the problems they will be asked to solve and about the questions that will appear on reading comprehension or recall tests; these expectations…

The purpose of this study was to compare the scores of students who were allowed unlimited retakes of a multiple-choice test with the scores of students who were limited to only four retakes (five trials) of the same test. The tests were each made up of 20 randomly drawn questions from a large pool of questions about research methods. Three…

This paper presents an overview of the design features that were developed for the Content Analysis Project. The purpose of the project was to examine the congruence between a state's test in eighth-grade mathematics and that used by the National Assessment of Educational Progress. The results of this analysis were then to be used to determine…

A context in which existing items may provide a convenient source of questions for questionnaires was explored through a case study making use of existing comparison groups. Two programs at Oak Ridge Associated Universities (ORAU), the Science and Engineering Research Semester (SERS) and the Laboratory Graduate Research Participation (Lab Grad)…

A practical application of generalizability theory, demonstrating how the variance components contribute to understanding and interpreting the data collected to evaluate a program, is described. The evaluation concerned 120 learning modules developed for the Dental Auxiliary Education Project. The goals of the project were to design, implement,…

Specific questions addressed in this study include how time limits affect a test's construct and predictive validities, how time limits affect an examinee's time allocation and test performance, and whether the assumption about how examinees answer items is valid. Interactions involving an examinee's sex and age are studied. Two parallel forms of…

A procedure for the detection of differential item performance (DIP) is used to investigate the relationships between characteristics of mathematics achievement items and gender differences in performance. Eight randomly equivalent samples of high school seniors were each given a unique form of the ACT Assessment Mathematics Usage Test (ACTM).…

The operational preparations for the Dutch national assessment program in education started in 1986. The program's aim is to periodically describe the contents of the curriculum in use and the attainment of 8-year-old and 11-year-old students on a national basis by means of surveys. In this paper some of the guiding principles in instrument…

The empirical validity of generalizability theory was investigated by applying two three-facet designs to data obtained in 1988 from administration of the Scientific Thinking and Research Skill Test (STRST). The decision validity of the STRST was also examined. Subjects were 125 fifth-grade and 125 sixth-grade students who were administered the…

Birnbaum's three-parameter logistic item response model was used to study guessing behavior of low ability examinees on the Graduate Record Examinations (GRE) General Test, Verbal Measure. GRE scoring procedures had recently changed, from a scoring formula which corrected for guessing, to number-right scoring. The three-parameter theory was used…

The purpose of this study was to provide teachers, supervisors, and school administrators with a valid scale for measuring teacher attitudes toward low achievers in mathematics: the Teacher Attitudes Toward Low Achievement in Mathematics Scale (TALAM). The development of the instrument was carried out in three phases. Phase 1 consisted of…