The purpose of this study was to investigate the effect of using multidimensional items in a computer adaptive test (CAT) setting which assumes a unidimensional item response theory (IRT) framework. Previous research has suggested that the composite of multidimensional abilities being estimated by a unidimensional IRT model is not constant…

The combinational-format item (CFI)--multiple-choice item with combinations of alternatives presented as response choices--was studied to determine whether CFIs were different from regular multiple-choice items in item characteristics or in cognitive processing demands. Three undergraduate Foundations of Education classes (consisting of a total of…

Describes advantages and disadvantages of computer-assisted foreign language testing, including unique capabilities of the computer, types of tests possible, limitations, problems, and language skill testing. The Spanish Computer Adaptive Placement Examination, developed at Brigham Young University (Utah), is described. (CB)

This paper reports on the results of a usability study that investigated the suitability of using an open systems approach for developing an online assessment application. Issues examined included ease of use, security, effectiveness, and ease of maintenance. Participants were 68 preservice teachers taking a technology integration course. Students…

This proof-of-concept investigation describes a computer-based approach for deriving the knowledge structure of individuals and of groups from their written essays, and considers the convergent criterion-related validity of the computer-based scores relative to human rater essay scores and multiple-choice test scores. After completing a…

This study seeks to determine whether item features are related to observed differences in item difficulty (DIF) between computer- and paper-based test delivery media. Examinees responded to 60 quantitative items similar to those found on the GRE general test in either a computer-based or paper-based medium. Thirty-eight percent of the items were…

This paper describes a study in which a Picture Readability Index (PRI) was used to investigate initial and extended perceptions of photographs. Readability criteria for evaluating instructional text seems to have been in place for a long time, yet instructional visuals like photographs and illustrations have typically been subject to no such…

The history of the Educational Testing Service (ETS) Factor Kits is summarized. The original ETS Factor Kit was developed in 1954 and contained 51 items, three each for each of 15 factors and six for a 16th factor. The next edition was developed in 1963 and included adaptations (clones) of the defining tests instead of the exact copies. These…

Reports on a study designed to determine whether or not biology students were being disadvantaged by taking computer tests instead of written paper forms of the same tests. Results indicated that mean scores on computer-administered and written forms of the same test were approximately equivalent. (TW)

Inconsistencies in scores from computer-simulated and "hands-on" science performance assessments have led to questions about the exchangeability of these two methods in spite of the highly touted potential of computer-simulated performance assessment. This investigation considered possible explanations for students' inconsistent performances: (1)…

Achievement testing in the next century is going to be very different. Computer technology is going to play a major role in test construction, test administration, scoring, and score reporting. New formats will become possible that incorporate visual and audio components and that permit adaption of tests to individual ability levels. (SLD)

This study examined performance of a simulated computerized adaptive test that was designed to help direct the development of a medical recertification examination. The item pool consisted of 229 single-best-answer items from a random sample of 3,000 examinees, calibrated using the two-parameter logistic model. Examinees' responses were known. For…

A predictive adaptive testing (PAT) strategy was developed based on statistical predictive analysis, and its feasibility was studied by comparing PAT performance to those of the Flexilevel, Bayesian modal, and expected a posteriori (EAP) strategies in a simulated environment. The proposed adaptive test is based on the idea of using item difficulty…

Examinees in a regular administration of the quantitative portion of the Graduate Record Examination responded to particular items in a machine-scannable multiple-choice format. Volunteers (n=364) used a computer to answer open-ended counterparts of these items. Scores for both formats demonstrated similar correlational patterns. (SLD)

This book offers specific how-to advice to college faculty on every stage of the testing process, including planning the test and classifying objectives to be measured, ensuring the validity and reliability of the test, and grading in such a way as to arrive at fair grades based on relevant data. The book examines the strengths and weaknesses of…