In this article, the author begins by discussing four challenges on the concept of validity. These challenges are: (1) the appeal to logic and syllogistic reasoning; (2) the claim of reliability; (3) the local and the universal; and (4) the unitary and the divisible. In language testing validity cannot be achieved directly but only through a…

This article presents the author's observations on Neil Dorans's NCME Career Award Address: "The Contestant Perspective on Taking Tests: Emanations from the Statue within." He calls attention to some points that Dr. Dorans made in his address, and offers his thoughts in response.

Neil Dorans has made a career of advocating for the examinee. He continues to do so in his NCME career award address, providing a thought-provoking commentary on some current trends in educational measurement that could potentially affect the integrity of test scores. Concerns expressed in the address call attention to a conundrum that faces…

In this speech, the author covers a lot of ground. In the first half of his speech, the author gives a brief summary of the last 40 years of the history of language testing, from his perspective. The author reviews these years more or less by decade. Additionally, he discusses the evolution of the profession of language testing during this period,…

Tests are a natural part of education, from the quizzes, essays, and classroom tests that teachers have traditionally administered to the high-stakes tests that states use to make decisions about graduation, promotion, and school funding and governance. In this article, the author stresses the need to learn the unintended consequences of…

This paper is the transcript of a talk given to those who use test information but who have little technical background in test theory. The concepts of modern test theory are compared with traditional test theory, as well as a probable future test theory. The explanations given are couched within an extended metaphor that allows a full description…

Points out the difficulty in judging a student's writing ability on the basis of one piece of writing. Suggests alternative evaluation procedures. (EL)

Presents simplified methods for deriving estimates of the consistency of criterion-referenced, English-as-a-Second-Language tests, including (1) the threshold loss agreement approach using agreement or kappa coefficients, (2) the squared-error loss agreement approach using the phi(lambda) dependability approach, and (3) the domain score…

Critically reviews an earlier published study that assessed the sensitivity of the cloze procedure to intersentential integration. (AEA)

Examines and contests the claim that all language tests are in reality testing the same underlying ability which is very similar to the Spearman g-factor for intelligence. Conclusions indicate that the argument for the existence of a g-factor in language tests is not tenable on statistical grounds. (SL)

Examines objectivity as a criterion for excellency in language tests through a critical presentation of the characteristics of objective tests. This examination leads to a discussion of what the best approach to testing language might be, endorsing a combination of integrative and objective techniques. (Author/MES)

The article focuses on the controversy over norm-referenced v criterion-referenced measures (CRM) in assessment of learning disorders. The authors contend that while the reliability of CRMs is generally indisputable, the validity of measures designed from local curricula is still dependent on the intuitive judgments of teachers. (Author/SBH)

There are two modes for testing writing: atomistic and holistic. Atomistic tests depend on isolating a form, emphasizing the recognition of the form rather than the creation of it. Holistic tests require the generation of writing, so that the forms are seen in a particular context. The atomistic tests have great danger of leading to false…

In light of the growing need for individuals to be assessed or assess themselves on workplace learning, the different notions of reliability and validity in classical test theory are examined as they might apply to work-based learning. It is concluded that, under certain stated conditions, test reliability, validity, and comparability can be…

Concerns about and objections to competency-based assessment in higher education are examined, including superficiality, use of inference, neglect of knowledge and of process, and subjectivity. It is concluded that, although each objection raises an important issue, in each case, a well-designed competency-based assessment system can overcome…