While score comparability across test forms typically relies on common (or randomly equivalent) examinees or items, innovations in item formats, test delivery, and efforts to extend the range of score interpretation may require a special data collection before examinees or items can be used in this way--or may be incompatible with common examinee…

Originally designed for students in Grades 3 through 5, MOCCA (formerly the Multiple-choice Online Causal Comprehension Assessment), identifies students who struggle with comprehension, and helps uncover why they struggle. There are many reasons why students might not comprehend what they read. They may struggle with decoding, or reading words…

Tatsuoka (1984) suggested several extended caution indices and their standardized versions that have been used as person-fit statistics by researchers such as Drasgow, Levine, and McLaughlin (1987), Glas and Meijer (2003), and Molenaar and Hoijtink (1990). However, these indices are only defined for tests with dichotomous items. This paper extends…

Students were observed directly, in a usability laboratory, and indirectly, by means of an extensive evaluation of responses, as they attempted interactive computer-marked assessment questions that required free-text responses of up to 20 words and as they amended their responses after receiving feedback. This provided more general insight into…

In almost all high-stakes testing programs, test equating is necessary to ensure that test scores across multiple test administrations are equivalent and can be used interchangeably. Test equating becomes even more challenging in mixed-format tests, such as Advanced Placement Program® (AP®) Exams, that contain both multiple-choice and constructed…

The purpose of this document is to provide background information that will be useful in interpreting the 2007 results from the Trends in International Mathematics and Science Study (TIMSS) by comparing its design, features, framework, and items with those of the U.S. National Assessment of Educational Progress and another international assessment…

The microcomputer version of the Ohio Vocational Interest Survey (OVIS II) differs from the machine-scored version in its ability to incorporate data from the OVIS II:Career Planner in its printed report. It differs from the hand-scored version in its ability to include data from the OVIS II:Work Characteristic Analysis in its printed report.…

The Student-Problem (S-P) methodology is described using an example of 24 students on a test of 44 items. Information based on the students' test score and the modified caution index is put to diagnostic use. A modification of the S-P methodology is applied to domain-referenced testing. (Author/CM)

Item overlap coefficient (IOC) formulas are discussed, providing six warnings about their calculation and interpretation and some explanations of why item overlap influences the Minnesota Multiphasic Personality Inventory and the Millon Clinical Multiaxial Inventory factor structures. (SLD)

A review of the research on item writing, item format, test instructions, and item readability indicated the importance of instrument structure in the interpretation of test data. The effect of failing to consider these areas on the content validity of achievement test scores is discussed. (Author/GK)

The Research Services of the National Education Association (NEA) conducted a nationwide teacher opinion poll (TOP) based upon a stratified disproportionate two-state cluster sample of classroom teachers. This research study was conducted to test the hypothesis that the order of presentation of items would make no difference in the conclusions…

The idea that validity should be considered a property of inferences, rather than of assessments, has developed slowly over the past century. In early writings about the validity of educational assessments, validity was defined as a property of an assessment. The most common definition was that an assessment was valid to the extent that it…

The Ohio Vocational Achievement Tests are specially designed instruments for use by teachers, supervisors, and administrators to evaluate and diagnose vocational achievement for improving instruction in secondary vocational programs at the 11th and 12th grade levels. This guide explains the Ohio Vocational Achievement Tests and how they are used.…

Standard and experimental forms of the Johnson O'Connor Research Foundations Analytical Reasoning test were administered to 1,496 clients of the Foundation (persons seeking information about aptitude for educational and career decisions). The objectives were to develop a new form of the test and to better understand what makes some items more…

Performance of 100 college freshmen on the Woodcock-Johnson Psycho-Educational Battery, Part II, Tests of Achievement, were analyzed by subtests and cluster scores to determine appropriateness for assessing achievement of handicapped students. Minor inversions in item order and pronounced ceiling effects on all subtests yielded lowered subtest and…