The purpose of the study was to determine comparability of an online version to the original paper-pencil version of Defining Issues Test 2 (DIT2). This study employed methods from both Classical Test Theory (CTT) and Item Response Theory (IRT). Findings from CTT analyses supported the reliability and discriminant validity of both versions.…

The relationship between levels of reliability and the power of two bias and differential item functioning (DIF) detection methods is examined. Both methods, the Mantel-Haenszel (MH) procedure of P. W. Holland and D. T. Thayer (1988) and the Simultaneous Item Bias (SIB) procedure of R. Shealy and W. Stout (1991), use examinees' raw scores as a…

A preliminary study is reported of the use of new multifaceted Rasch measurement mechanisms for investigating rater characteristics in language testing. Ratings from four judges of scripts from 50 candidates taking the International English Language Testing System test, a test of English for Academic Purposes, are analyzed. The analysis…

This paper explores the comparability of item calibrations for three types of items: (1) text only; (2) text with photographs; and (3) text plus graphics when items are presented on written tests and computerized adaptive tests. Data are from five different medical technology certification examinations administered nationwide in 1993. The Rasch…

To determine the most accurate group-administered measure of reading achievement, a study explored variations of the cloze and maze procedures with second grade students who were native English speakers or who were being taught English as a second language. Subjects--108 second grade volunteers (1% American Indian, 49% Asian, 39.8% Black, 1%…

A study assessed the reliability and some aspects related to the validity of an open sentence approach to assessing comprehension during silent reading. Subjects, 37 randomly selected seventh-grade students, read an "Opin" passage, prepared by deleting the second half of every other sentence. The blanks filled in by students were…

A comparison was made of Durrell Analysis of Reading Difficulty and informal reading inventory (IRI) independent and instructional level designations for 15 children, aged 9 to 11 years, who had been referred to a diagnostic clinic for reading assessment. The children's reading performance was first scored according to procedures outlined in the…

This report describes a study comparing the classification results obtained from a one-parameter and three-parameter logistic based tailored testing procedure used in conjunction with Wald's sequential probability ratio test (SPRT). Eighty-eight college students were classified into four grade categories using achievement test results obtained…

Procedures for determining cutting scores have been proposed by Angoff and by Nedelsky. Nedelsky's approach requires that a rater examine each distractor within a test item to determine the probability of a minimally competent examinee answering correctly; whereas Angoff uses a judgment based on the whole item, rather than each of its components.…

The precision and efficiency of a cognitive test constructed by three different methods of item analysis was compared, using the verbal aptitude subtest of the Florida Twelfth Grade Test. Classical item analysis, factor analysis and the Rasch logistic model were used in the construction of 15 and 30 item subtests and replicated for samples of 250,…

Since the accuracy and validity of occupational data may vary according to the rating scale format employed, the first phase of the research described in the report employed hypothetical job descriptions from which accurate criterion data could be generated. The second phase of the research required developing an occupational survey instrument…

The purpose of this study was to explore the usefulness of the Rasch test analysis model for providing consistent scale statistics between content-referenced and norm-referenced tests across school districts. A number of content-referenced tests were created to assess student performance relative to basic skills goals identified in reading and…

The construction of an exploratory test dealing with concepts relevant to mathematics was required as part of the New Canadian Study. The test devised for this purpose has three sections dealing with the sorting of objects. In Sort One, 10 lines are sorted into groups of two; in Sort Two, numbers from 1 to 10 are sorted; and, in Sort Three,…

Concludes that the Multi-Digit Test stimulates better retention than multiple choice tests while offering the advantage of computerized scoring and analysis. (FL)

Term papers written by 42 students in two subjects were assessed by their classmates and by lecturers in the relevant subjects. No difference occurred between lecturers and students in average marks, variations in marks, agreement in marks (reliability), or relationship between marks and performance in final examinations (validity). (Author/MSE)