This report describes technical plans and progress during the second year (Fiscal Year 1984) of work on the U.S. Army's Project A: "Improving the Selection, Classification, and Utilization of Army Enlisted Personnel." Project A is a long-range research program developed by the Army Research Institute for the Behavioral and Social…

This report explores an approach to item development and psychometric modeling which explicitly incorporates knowledge about the mental models used by examinees in the solution of items into a psychometric model that characterize performances on a test, as well as incorporating that knowledge into the item development process. The paper focuses on…

Limitations seen in traditional educational procedures are discussed, and three new directions are suggested as being important emphases for testing. A new perception of measurement in which children have positions along particular concept dimensions or "lines" is discussed in terms of traditional testing notions including variables, item banks…

The feasibility of creating an item bank from a teacher-made test was examined in two comparable sections of a graduate-level introductory measurement course. The 67-item midterm examination contained multiple-choice and master matching items, which required higher level cognitive processes such as application and analysis. The feasibility of…

A live tailored achievement testing study was conducted to compare procedures based on the one- and three-parameter logistic models. Previous studies yielded inconclusive results because of the procedures by which item calibrations were linked and because of the item selection procedures. Using improved procedures, 83 college students were tested…

During the past few years, several Harvard paper surveys were converted to Web surveys. These were high-profile surveys endorsed by the Provost and the Dean of the College, and covered major portions of the university population (all undergraduates, all graduate students, tenured and non-tenured faculty). When planning for these surveys started in…

Technology today offers many new opportunities for innovation in educational assessment through rich new assessment tasks and potentially powerful scoring, reporting and real-time feedback mechanisms. One potential limitation for realizing the benefits of computer-based assessment in both instructional assessment and large scale testing comes in…

This paper draws attention to literature surrounding the subject of computer-assisted assessment (CAA). A brief overview of traditional methods of assessment is presented, highlighting areas of concern in existing techniques. CAA is then defined, and instances of its introduction in various educational spheres are identified, with the main focus…

Employment interviews are widely used in the selection of quality teachers, and indeed research confirms administrators' belief in the validity of the procedure. However, many key recommendations for improving the general reliability of interviews including selecting questions that are job-related and research grounded, including well designed…

In this technical report, we describe the development and piloting of a series of mathematics progress monitoring measures intended for use with students in grades kindergarten through eighth grade. These measures, available as part of easyCBM[TM], an online progress monitoring assessment system, were developed in 2007 and 2008 and administered to…

The Raggio Evaluation of Attention Deficit Disorder (READD) is an objective measure for the diagnosis and management of attention deficit disorder (ADD) in children. Extensive research has been conducted on its clinical and psychometric properties, as described in Chapter 3, "Development and Standardization." The READD is a microcomputer…

Testbanking provides teachers with an effective, low-cost, time-saving opportunity to improve the testing aspect of their classes. Testbanking, which involves the use of a testbank program and a computer, allows teachers to develop and generate tests and test-forms with a minimum of effort. Teachers who test using true and false, multiple choice,…

This study was intended to establish "base-line" data with respect to teacher utilization of available microcomputer software for the purposes of: (1) generating teacher-made appraisals; (2) scoring/analyzing teacher-made appraisals; and (3) assigning and recording grades. Differences in reported utilization at the K-4, 5-8, and 9-12 grade levels…

The purpose of this study was to investigate the effect of using multidimensional items in a computer adaptive test (CAT) setting which assumes a unidimensional item response theory (IRT) framework. Previous research has suggested that the composite of multidimensional abilities being estimated by a unidimensional IRT model is not constant…

An instrument was developed to measure students' engagement in various classes of study activities within multiple study contexts. This instrument is to be used in a large, ongoing research project investigating the underpinnings of student study activities and effects on academic achievement. The computer-administered instrument used--the Study…