Validity is one of the psychometric properties of the achievement tests. To determine the validity, one of the examination is item bias studies, which are based on differential item functioning (DIF) analyses and field experts' opinion. In this study, field experts were asked to estimate the DIF levels of the items to compare the estimations…

Test scores matter these days. Test-takers want to understand how they performed, and test score reports, particularly those for individual examinees, are the vehicles by which most people get the bulk of this information. Historically, score reports have not always met the examinees' information or usability needs, but this is clearly changing…

Many credentialing agencies today are either administering their examinations by computer or are likely to be doing so in the coming years. Unfortunately, although several promising computer-based test designs are available, little is known about how well they function in examination settings. The goal of this study was to compare fixed-length…

Computer-based testing by credentialing agencies has become common; however, selecting a test design is difficult because several good ones are available - parallel forms, computer adaptive (CAT), and multistage (MST). In this study, three computer-based test designs under some common examination conditions were investigated. Item bank size and…

This paper offers answers to nine important questions concerning standard-setting issues and methods: (1) Should normative or content-referenced standards be used? (2) Different standard-setting methods yield different results. What is your reaction to this finding? That of your clients? Does this finding present a problem for the application of…

The development and evaluation of methods for detecting potentially biased items or differentially functioning items (DIF) represent a critical area of research for psychometricians because of the negative impact of biased items on test validity. A summary is provided of the authors' 12 years of research at the University of Massachusetts…

Authentic measurement has become an important topic recently in educational testing. Advocates of authentic measurement feel that objective tests, multiple-choice tests in particular, cannot meet the demands required of today's tests and should be replaced by tests that can be closely matched to instruction and can assess higher-order cognitive…

Increasing social awareness of the need for accurate and fair testing has combined with technological developments to produce dramatic advances in psychometric models, methods, and procedures. Three of the most important new directions within the measurement field concern the following areas: (1) item response theory; (2) item bias or differential…

Comments are made on the review papers presented by six Dutch psychometricians: Ivo Molenaar, Wim van der Linden, Ed Roskam, Arnold Van den Wollenberg, Gideon Mellenbergh, and Dato de Gruijter. Molenaar has embraced a pragmatic viewpoint on Bayesian methods, using both empirical and pure approaches to solve educational research problems. Molenaar…

The problem of determining optimal test lengths with fixed total testing time has proved to be a difficult one for criterion-referenced test developers. An algorithm is needed which can be used by test developers to allocate available testing time to maximize the validity of their total criterion-referenced tests or testing programs. To be…

The validity of several criticisms of objective tests is addressed, and the viability of some alternatives to objective testing is discussed. Evidence against multiple-choice tests is not as strong as has been claimed. Authentic assessments may not always be better, and research about new forms of assessment is necessary. (SLD)

The success of objectives-based programs depends to a considerable extent on how effectively students and teachers assess mastery of objectives and make decisions for future instruction. While educators disagree on the usefulness of criterion-referenced tests the position taken in this monograph is that criterion-referenced tests are useful, and…

This paper highlights some special characteristics of criterion-referenced tests while comparing them with norm-referenced tests. Psychometric considerations involved in constructing a criterion-referenced test, including item analysis, reliability, and validity, are discussed, along with application of criterion-referenced testing to individual…