The current paper on writing assessment surveys the literature on the reliability and validity of essay tests. The paper aims to examine the two concepts in relationship with essay testing as well as to provide a snapshot of the current understandings of the reliability and validity of essay tests as drawn in recent research studies. Bearing in…

The major purpose of this study is to examine the quality of ratings assigned to CR (constructed-response) questions in large-scale assessments from the perspective of Rasch Measurement Theory. Rasch Measurement Theory provides a framework for the examination of rating scale category structure that can yield useful information for interpreting the…

In 1997, based on the amendments to Individuals with Disabilities Education Act (IDEA), all states were faced with a statutory requirement to develop and implement alternate assessments for students with disabilities unable to participate in the statewide large-scale assessment. States were given the challenge of creating, implementing, and…

Any examination that involves moderate to high stakes implications for examinees should be psychometrically sound and legally defensible. Currently, there are two broad and competing families of test theories that are used to score examination data. The majority of instructors outside the high-stakes testing arena rely on classical test theory…

Views on testing--its purpose and uses and how its data are analyzed--are related to one's perspective on test takers. Test takers can be viewed as learners, examinees, or contestants. I briefly discuss the perspective of test takers as learners. I maintain that much of psychometrics views test takers as examinees. I discuss test takers as a…

"Test Scoring" provides insight into psychometric procedures as used by a professional testing company or in large-scale projects. The book contains an overview of standard test theory, a discussion of factor analytic theory, and an exploration of special applications and problems. (SLD)

This paper addresses practical Army problems in reading assessment from a theory base reflecting the most recent research on reading comprehension. Military and occupational research shows that reading proficiency is related to job performance. Reading assessment is a key issue in the Army due to changes in the reading ability levels of the Army…

Significant to a latent trait or item response theory analysis of a mental test is the determination of exactly what is being quantified. The following are practical problems to be considered in the formulation of a good theory: (1) deciding whether two tests measure the same trait or traits; (2) analyzing the relative contributions of a pair of…

Multiple-choice testing (MCT) has several advantages which are becoming more relevant in the current financial climate. In particular, they can be machine marked. As an objective testing method it is particularly relevant to engineering and other factual courses, but MCTs are not widely used in engineering because students can benefit from…

In an adaptive test, the test administrator chooses test items sequentially during the test, in such a way as to adapt test difficulty to examinee ability as shown during testing. An effectively designed adaptive test can resolve the dilemma inherent in conventional test design. By tailoring tests to individuals, the adaptive test can…

This text was written to help the reader acquire a base of knowledge about classical psychometrics and to integrate new ideas into that framework of knowledge. The material is organized into five units: (1) introduction to measurement theory; (2) reliability; (3) validity; (4) item analysis in test development; and (5) test scoring and…

This digest presents a list of recommendations for writing multiple-choice test items, based on psychometrics and logical deduction. Questions should ask more than mere knowledge of facts and should not contain superfluous information as an introduction to the question. Each question should focus on some specific aspect of the course, and the item…

Although perfectly scalable items rarely occur in practice, Guttman's concept of a scale has proved to be valuable to the development of measurement theory. If the score distribution is uniform and there is an equal number of items at each difficulty level, both the elements and the eigenvalues of the Pearson correlation matrix of dichotomous…

The theoretical framework and cross-cultural validation of Tell-Me-A-Story (TEMAS), a projective test developed to measure personality development in ethnic minority children, is presented. The TEMAS test consists of 23 chromatic pictures which incorporate the following characteristics: (1) representation of antithetical concepts which the…