This report describes statistical analyses performed on data collected in the Spring of 2015 from the pilot study of proposed revised and new items in the IDEA Student Ratings of Instruction (SRI) system. Described are the methods employed, results obtained, and decisions made in selecting items for the updated instruments. The procedures occurred…

This report describes the processes undertaken to revise the IDEA Student Ratings of Instruction (SRI) system. The previous revision occurred in 1999, as described in IDEA Technical Report No. 11, "Revising the IDEA System for Obtaining Student Ratings of Instructors and Courses" (Hoyt, Chen, Pallett, & Gross, 1999). The procedures…

In this technical report, we present the results of a reliability study of the seventh-grade multiple choice reading comprehension measures available on the easyCBM learning system conducted in the spring of 2011. Analyses include split-half reliability, alternate form reliability, person and item reliability as derived from Rasch analysis,…

The purpose of this study was to develop general outcome measures (GOM) in mathematics so that teachers could focus their instruction on needed prerequisite skills. We describe in detail, the manner in which content-related evidence was established and then present a number of statistical analyses conducted to evaluate the technical adequacy of…

Some problems associated with the criteria of reproducibility and scalability as they are used in Guttman scalogram analysis to evaluate cumulative, nonparametric scales of dichotomous items are discussed. A computer program is presented which analyzes response patterns elicited by dichotomous scales designed to be cumulative. (Author/DWH)

This technical report describes the development of reading comprehension assessments designed for use as progress monitoring measures appropriate for 2nd Grade students. The creation, piloting, and technical adequacy of the measures are presented. The following are appended: (1) Item Specifications for MC [Multiple Choice] Comprehension - Passage…

This technical report describes the development and piloting of reading comprehension measures developed for use by fifth-grade students as part of an online progress monitoring assessment system, http://easycbm.com. Each comprehension measure is comprised of an original work of narrative fiction approximately 1500 words in length followed by 20…

It is a widely held belief that anchor tests should be miniature versions (i.e., minitests), with respect to content and statistical characteristics of the tests being equated. This paper examines the foundations for this belief. It examines the requirement of statistical representativeness of anchor tests that are content representative. The…

The "2004/06 Beginning Postsecondary Students Longitudinal Study (BPS:04/06)" is sponsored by the U.S. Department of Education to respond to the need for a national, comprehensive database concerning issues students may face in enrollment, persistence, progress, and attainment in postsecondary education and in consequent early rates of…

Fit statistics provide a direct measure of assessment accuracy by analyzing the fit of measurement models to an individual's (or group's) response pattern. Students that lose interest during the assessment, for example, will miss exercises that are within their abilities. Such students will respond correctly to some more difficult items and…

This paper presents a study on the generation of mathematics test items using algorithmic methods. The history of this approach is briefly reviewed and is followed by a survey of the research to date on the statistical parallelism of algorithmically generated mathematics items. Results are presented for 8 parallel test forms generated using 16…

The impact of allowing more time for each question on SAT® I: Reasoning Test scores was estimated by embedding sections with a reduced number of questions into the standard 30-minute equating section of two national test administrations. Thus, for example, questions were deleted from a verbal section that contained 35 questions to produce forms…

Differential item functioning (DIF) occurs when the responses of students with approximately equal ability differ systematically based on their group membership. Idaho contracted with the Northwest Evaluation Association (NWEA) to provide the Idaho Standards Achievement Tests of reading, language arts and mathematics at grades 4, 8 and 10. The…

This paper describes the development, implementation, and evaluation of an automated system for predicting the acceptability status of candidate reading-comprehension stimuli extracted from a database of journal and magazine articles. The system uses a combination of classification and regression techniques to predict the probability that a given…

This report describes the methods and procedures used for the Beginning Postsecondary Students Longitudinal Study: 1996-2001 (BPS:1996/2001). These students, who started their postsecondary education during the 1995-96 academic year, were first interviewed in 1996 as part of the National Postsecondary Student Aid Study (NPSAS:96). A follow-up…