Three pyramidal adaptive tests and a conventional peaked test were constructed and administered by computer to two groups of students enrolled in undergraduate psychology courses. Six methods of scoring pyramidal tests were evaluated with respect to score distributions, stability, and the degree of relationship among scoring methods and between…

Issues that must be addressed in the large-scale application of computerized adaptive testing are explored, including considerations of test design, scoring, test administration, item and item bank development, and other aspects of test construction. Possible solutions and areas in which additional work is needed are identified. (SLD)

C. Davis and M. Cowles (1989) analyzed a total trait anxiety score on the Endler Multidimensional Anxiety Scales (EMAS)--a unidimensional construct that this multidimensional measure does not assess. Data are reanalyzed using the appropriate scoring procedure for the EMAS. Subjects included 145 undergraduates in 1 of 4 testing conditions. (SLD)

Reports the results of several test-reliability experiments which compared a modified confidence weighted-admissible probability measurement (MCW-APM) with conventional forced choice or binary type (R-W) test scoring methods. Psychometric properties using G theory and conventional correlational methods are examined, and their implications for…

Describes the use of interactive videodiscs in Dutch secondary vocational school departments of pharmaceutical education for testing theoretical knowledge and practical skills in a simulated real-life situation. An example is given, feedback and scoring are explained, and criteria for reliability with a classical text analysis are discussed.…

This study examined the construct validity of the "e-rater"® automated essay scoring engine as an alternative to human scoring in the context of TOEFL® essay writing. Analyses were based on a sample of students who repeated the TOEFL within a short time period. Two "e-rater" scores were investigated in this study, the first…

There is a considerable interest at Educational Testing Service (ETS) to include performance-based, natural language constructed-response items on standardized tests. Such items can be developed, but the projected time and costs required to have these items scored by human graders would be prohibitive. In order for ETS to include these types of…

The Quest program offers a comprehensive test and questionnaire analysis environment by providing a data analyst (a computer program) with access to the most recent developments in Rasch measurement theory, as well as a range of traditional analysis procedures. This manual helps the user use Quest to construct and validate variables based on…

This handbook provides teachers and administrators with strategies to construct, adapt, and use rubrics. A rubric is defined as a tool for assessing instruction and performance according to predetermined expectations and criteria. The chapters in this text contain guidance on formulating, applying, and reviewing the pros and cons of this form of…

The concept of testing for partial knowledge is considered with the concept of tailored testing. Following the special usage of latent trait theory, the word valdity is used to mean the correlation of a test with the construct the test measures. The concept of a method factor in the test is also considered as a part of the validity. The possible…

This report documents the technology initiatives of the Center for Research on Evaluation, Standards, and Student Testing (CRESST) in two broad areas: (1) using technology to improve the quality, utility, and feasibility of existing measures; and (2) using technology to design and develop new assessments and measurement approaches available…

Describes an automated scoring algorithm for a computer-based simulation examination of physicians' patient-management skills. Results with 280 medical students show that scores produced using this algorithm are highly correlated to actual clinician ratings. Scores were also effective in discriminating between case performance judged passing or…

Maximum likelihood and Bayesian procedures are presented for item selection and scoring of multidimensional adaptive tests. A demonstration with simulated response data illustrates that multidimensional adaptive testing can provide equal or higher reliabilities with fewer items than are required in one-dimensional adaptive testing. (SLD)

Latent semantic analysis (LSA) is an automated, statistical technique for comparing the semantic similarity of words or documents. In this article, I examine the application of LSA to automated essay scoring. I compare LSA methods to earlier statistical methods for assessing essay quality, and critically review contemporary essay-scoring systems…

This study was designed to address two frequent criticisms of the SAT essay--that essay length is the best predictor of scores, and that there is an advantage in using more "sophisticated" examples as opposed to personal experience. The study was based on 2,820 essays from the first three administrations of the new SAT. Each essay was…