This article reports on the Pittsburgh Public Schools Oral Ratings Assessment for Language Students (PPS ORALS) project, a Title VI Foreign Language Assistance Program grant-funded project to create an online testing Software program that makes large-scale oral testing in a world language not only feasible, but also easy to create, administer, and…

Podcasting has become a popular medium for accessing and assimilating information and podcasts are increasingly being used to deliver audio recordings of lectures to campus-based students. This paper describes a simple, cost-effective and file size-efficient method for producing video podcasts combining lecture slides and audio without a…

In this technical report, we describe the development and piloting of a series of mathematics progress monitoring measures intended for use with students in grades kindergarten through eighth grade. These measures, available as part of easyCBM[TM], an online progress monitoring assessment system, were developed in 2007 and 2008 and administered to…

As GED (General Educational Development) Testing Service considers the feasibility of a computer administration of the GED Tests, one issue being considered is the difference in costs of supplying only a computer-based format vs. offering both computer-based and paper formats of the GED Tests. A significant concern then arises as to whether…

This report provides a two-part evaluation of the IntelliMetric[SM] automated essay scoring system based on its performance scoring essays from the Analytic Writing Assessment of the Graduate Management Admission Test[TM] (GMAT[TM]). The IntelliMetric system performance is first compared to that of individual human raters, a Bayesian system…

When an examination consists, in whole or in part, of constructed response items, it is a common practice to allow the examinee to choose among a variety of questions. This procedure is usually adopted so that the limited number of items that can be completed in the allotted time does not unfairly affect the examinee. This results in the de facto…

This paper discusses a computer-based prototype called TestMaker that enables educators to create computer-based tests. Given the functional needs of faculty, the host of research implications computer technology has for assessment, and current educational perspectives such as constructivism and their impact on testing, the purposes for developing…

Examinee response times from a computerized adaptive test taken by 204 examinees taking a certification examination were analyzed using a hierarchical linear model. Two equations were posed: a within-person model and a between-person model. Variance within persons was eight times greater than variance between persons. Several variables…

This report summarizes the results from two studies. The first assessed the comparability of scores derived from linear computer-based (CBT) and computer adaptive (CAT) versions of the three Graduate Record Examinations (GRE) General Test measures. A verbal CAT was taken by 1,507, a quantitative CAT by 1,354, and an analytical CAT by 995…

There is a considerable interest at Educational Testing Service (ETS) to include performance-based, natural language constructed-response items on standardized tests. Such items can be developed, but the projected time and costs required to have these items scored by human graders would be prohibitive. In order for ETS to include these types of…

Most use of multimedia technology in teaching and learning to date has emphasized the teaching aspect only. An application of multimedia in examinations has been neglected. This paper addresses how multimedia technology can be applied to the automatization of assessment, by proposing a prototype of a multimedia question bank, which is able to…

Differences in test performance on time-limited tests may be due in part to differential response-time rates between subgroups, rather than real differences in the knowledge, skills, or developed abilities of interest. With computer-administered tests, response times are available and may be used to address this issue. This study investigates…

At Humber College (HC) in Toronto, Ontario, Canada, the Digital Electronics (DE) program utilizes a computerized learning infrastructure called Computer Managed Learning (CML). The program, which has been under development for several years, is flexible enough to build a unique program of studies for each individual student and allows for the…

One important and promising application of item response theory (IRT) is computerized adaptive testing (CAT). The implementation of a nominal response model-based CAT (NRCAT) was studied. Item pool characteristics for the NRCAT as well as the comparative performance of the NRCAT and a CAT based on the three-parameter logistic (3PL) model were…

This paper reports on existing studies and uses meta analysis to compare and synthesize the results of 20 studies from 8 research reports comparing the ability measure equivalence of computer adaptive tests (CAT) and conventional paper and pencil tests. Using the research synthesis techniques developed by Hedges and Olkin (1985), it is possible to…