Twelve structural models of similarity were fitted to data from conventional and computer adaptive test (CAT) batteries measuring the same aptitude in a double cross-validation design. Three of the 12 models, including a multiplicative structure model, performed well, providing support for using CATs as replacements for conventional tests. (NSF)

Argues that, while computers make the computation of readability formulas easy, teachers should not forget the need to apply judgment and common sense to the results. Discusses RIXRATE, a computerized version of the Rix index and compares its performance to that of the Rauding Scale. (FL)

Explores the potential for using computers in the diagnosis of learning disabilities. Describes a prototype computer based system designed to assist teachers in the assessment of reading difficulties. (FL)

This paper presents a multistage adaptive testing test development paradigm that promises to handle content balancing and other test development needs, psychometric reliability concerns, and item exposure. The bundled multistage adaptive testing (BMAT) framework is a modification of the computer-adaptive sequential testing framework introduced by…

This study examined the time different ability level examinees spend taking a CAT on demanding items to these examinees. It was also found that high able examinees spend more time on the pretest items, which are not tailored to the examinees' ability level, than do lower ability examinees. Higher able examinees showed persistence with test…

When low-stakes assessments are administered to examinees, the degree to which examinees give their best effort is often unclear, complicating the validity and interpretation of the resulting test scores. This study introduces a new method for measuring examinee test-taking effort on computer-based test items based on item response time. This…

The purpose of this project was to develop valid evaluation procedures for the selection of Container Equipment Operators (CEOs) in the shipping industry. A job analysis was conducted to identify the essential tasks of the CEO job. Site visits, a task inventory, and the determination of essential tasks were used in the job analysis. The skills and…

One challenge associated with computerized adaptive testing (CAT) is the maintenance of test and item security while allowing for daily testing. An alternative to continually creating new pools containing an independent set of items would be to consider each CAT pool as a sample of items from a larger collection (referred to as a VAT) rather than…

Many large-scale testing programs routinely pretest new items alongside operational (or scored) items to determine their empirical characteristics. If these pretest items pass certain statistical criteria, they are placed into an operational item pool; otherwise they are edited and re-pretested or simply discarded. In these situations, reliable…

This report documents progress at the Center for Research on Evaluation, Standards, and Student Testing (CRESST) on the feasibility of scoring concept maps using technology. CRESST, in its integrated simulation approach to assessment, has assembled a suite of performance assessment tasks (the integrated simulation) onto which they have mapped the…

A conception of discussing mathematical material in the domain of calculus is outlined. Applications include that university students work at their knowledge and prepare for their oral examinations by utilizing the dialog system. The conception is based upon three pillars. One central pillar is a knowledge base containing the collections of…

This study explored the effect of item exposure on two conventional examinations administered as computer-based tests. A principal hypothesis was that item exposure would have little or no effect on average difficulty of the items over the course of an administrative cycle. This hypothesis was tested by exploring conventional item statistics and…

Recently, the Test of English as a Foreign Language (TOEFL) changed by including a writing section that gives the examinee an option between computer and handwritten formats to compose their responses. Unfortunately, this may introduce several potential sources of error that might reduce the reliability and validity of the scores. The seriousness…

Recently, the Test of English as a Foreign Language (TOEFL) changed by including a direct writing assessment where examinees choose between computer and handwritten composition formats. Unfortunately, examinees may have differential access to and comfort with computers; as a result, scores across these formats may not be comparable. Analysis of…

A constrained computerized adaptive testing (CAT) algorithm is presented that automatically equates the number-correct scores on adaptive tests. The algorithm can be used to equate number-correct scores across different administrations of the same adaptive test as well as to an external reference test. The constraints are derived from a set of…