This report presents factors to consider in the design of computer-based testing for all students, including students with disabilities and students with limited English proficiency. It also provides a process for the initial transformation of paper/pencil assessments to inclusive computer-based testing. Steps include: (1) assemble a group of…

The requirements of the revised Elementary and Secondary School Act for annual assessment of student progress highlight problems in using tests in a stringent manner to assess students and teachers. Issues such as validity, reliability, and time spent on test preparation must be considered. Some tests are used for prediction of future performance,…

This paper discusses six challenges relevant to Web-based testing. Some of these challenges are not specific to Web-based testing, but generalize to all computer-based testing. The challenges are: (1) security and using test centers for Web-based testing; (2) measuring complex skills and problem-solving tasks on the Web; (3) integrating modern…

This study investigated specific conditions under which out-of-scale information improves measurement precision and the factors that influence the degree of reliability gains and the amount of bias induced in the reported scores when out-of-scale information is used. In-scale information is information that an item provides for a composite trait…

This paper addresses instrumentation design and an ongoing study of an online formative assessment instrument and its effects in pre-college (developmental/remedial) mathematics courses. The use of facet theory and instructional technologies are harmonized to construct and scale an online instrument designed to document student procedures while…

Because of the importance of beliefs, mathematics educators need to consider ways to assess beliefs and belief change. Beliefs, because they must be inferred, can be difficult to measure, particularly with a common metric that enables one to compare individuals. Some limitations of Likert scales are identified and overcome in a newly developed…

Large-scale testing programs are generally based on the assumptions that the test-takers experience standard conditions for taking the test and that everyone will do his or her own work without having prior knowledge of specific questions. These assumptions are not necessarily true. The ways students and educators use to get around standardizing…

This report examines issues in the scale-up of assessment software from the Center for Research on Evaluation, Standards, and Student Testing (CRESST). "Scale-up" is used in a metaphorical sense, meaning adding new assessment tools to CRESST's assessment software. During the past several years, CRESST has been developing and evaluating a…

Procedures used to establish the comparability of scores derived from the College Board Admissions Testing Program (ATP) computer adaptive Scholastic Aptitude Test (SAT) prototype and the paper-and-pencil SAT are described in this report. Both the prototype, which is made up of Verbal and Mathematics computer adaptive tests (CATs), and a form of…

The purpose of this study was to determine if a relationship exists between scores and the times that medical students choose to take a computer-administered test. The results indicate that students who choose to take a test later within a given time period tend to perform less well than students who take the test earlier. Although the magnitude…

The purpose of this study was to compare and evaluate five online pretest item calibration/scaling methods in computerized adaptive testing (CAT): (1) the marginal maximum likelihood estimate with one-EM cycle (OEM); (2) the marginal maximum likelihood estimate with multiple EM cycles (MEM); (3) Stocking's Method A (M. Stocking, 1988); (4)…

An essay test is now an integral part of the computer based Test of English as a Foreign Language (TOEFL-CBT). This paper provides a brief overview of the current TOEFL-CBT essay test, describes the operational procedures for essay scoring, including the Online Scoring Network (OSN) of the Educational Testing Service (ETS), and discusses major…

This paper provides a technology-based review of World Wide Web-based testing technologies. It suggests an evaluation framework that could be used by practitioners in Web-based education to understand and compare features available in various Web-based testing systems. In order to compare existing options, the life cycle of a question in Web-based…

Two operational item pools were used to investigate the performance of stratum computerized adaptive tests (CATs) when items were assigned to strata based on empirical estimates of item difficulty or human judgments of item difficulty. Items from the first data set consisted of 54 5-option multiple choice items from a form of the ACT mathematics…

Error indices (bias, standard error of estimation, and root mean square error) obtained on different scales of measurement under different test termination rules in a computerized adaptive test (CAT) context were examined. Four ability estimation methods were studied: (1) maximum likelihood estimation (MLE); (2) weighted likelihood estimation…