This report summarizes the results from two studies. The first assessed the comparability of scores derived from linear computer-based (CBT) and computer adaptive (CAT) versions of the three Graduate Record Examinations (GRE) General Test measures. A verbal CAT was taken by 1,507, a quantitative CAT by 1,354, and an analytical CAT by 995…

There is a considerable interest at Educational Testing Service (ETS) to include performance-based, natural language constructed-response items on standardized tests. Such items can be developed, but the projected time and costs required to have these items scored by human graders would be prohibitive. In order for ETS to include these types of…

Most use of multimedia technology in teaching and learning to date has emphasized the teaching aspect only. An application of multimedia in examinations has been neglected. This paper addresses how multimedia technology can be applied to the automatization of assessment, by proposing a prototype of a multimedia question bank, which is able to…

Differences in test performance on time-limited tests may be due in part to differential response-time rates between subgroups, rather than real differences in the knowledge, skills, or developed abilities of interest. With computer-administered tests, response times are available and may be used to address this issue. This study investigates…

At Humber College (HC) in Toronto, Ontario, Canada, the Digital Electronics (DE) program utilizes a computerized learning infrastructure called Computer Managed Learning (CML). The program, which has been under development for several years, is flexible enough to build a unique program of studies for each individual student and allows for the…

One important and promising application of item response theory (IRT) is computerized adaptive testing (CAT). The implementation of a nominal response model-based CAT (NRCAT) was studied. Item pool characteristics for the NRCAT as well as the comparative performance of the NRCAT and a CAT based on the three-parameter logistic (3PL) model were…

This paper reports on existing studies and uses meta analysis to compare and synthesize the results of 20 studies from 8 research reports comparing the ability measure equivalence of computer adaptive tests (CAT) and conventional paper and pencil tests. Using the research synthesis techniques developed by Hedges and Olkin (1985), it is possible to…

Two studies investigated the comparability of scores from paper-and-pencil and computer-administered versions of the College-Level Examination Program (CLEP) General Examinations in mathematics and English composition. The first study used a prototype computer-administered version on each examination for 94 students for mathematics and 116 for…

Little research exists on the psychological impacts of computerized adaptive testing (CAT) and how it may affect test performance. Three CAT procedures were examined, in which items were selected to match students' achievement levels, from the item pool at random, or according to student choice of item difficulty levels. Twenty-four graduate…

A computerized adaptive test for placement of students in postsecondary French second language courses is evaluated for unidimensionality of its three component tests: reading comprehension of a short paragraph; selection of the appropriate statement in a given situation; and a "fill-in-the-blank" section. A variety of statistical…

The purpose of this research was to compare two different forms of a placement test in French as a second language at the post-secondary level, a conventional test and a computerized test. First, expectations regarding a placement test as a measure of general proficiency are discussed and principles of adaptive testing that have been used to…

The Quest program offers a comprehensive test and questionnaire analysis environment by providing a data analyst (a computer program) with access to the most recent developments in Rasch measurement theory, as well as a range of traditional analysis procedures. This manual helps the user use Quest to construct and validate variables based on…

A two-stage testing strategy is one method of adapting the difficulty of a test to an individual's ability level in an effort to achieve more precise measurement. A routing test provides an initial estimate of ability level, and a second-stage measurement test then evaluates the examinee further. The measurement accuracy and efficiency of item…

This paper offers a scenario for how educational assessment might change in response to market forces that affect not only the future of large-scale testing but also society in general. The scenario divides into three generations distinguished by the purpose of testing, test format and content, and the extent to which testing capitalizes on new…

This handbook provides teachers and administrators with strategies to construct, adapt, and use rubrics. A rubric is defined as a tool for assessing instruction and performance according to predetermined expectations and criteria. The chapters in this text contain guidance on formulating, applying, and reviewing the pros and cons of this form of…