Rater training is an important part of developing and conducting large-scale constructed-response assessments. As part of this process, candidate raters have to pass a certification test to confirm that they are able to score consistently and accurately before they begin scoring operationally. Moreover, many assessment programs require raters to…

The number of public school students who are English learners (ELs) has been increasing steadily in the United States. According to a report by Snyder, de Brey, and Dillow (2016), in the 2014-2015 school year, nearly 4.6 million students enrolled in Kindergarten through grade 12 (K-12) in U.S. schools were English learners, representing…

Examined multidimensional assessment of career decision problems as measured by Career Decision Diagnostic Assessment (CDDA). Administered CDDA to 844 college students from 3 universities. Results supported reliability and validity of CDDA as measure of psychological blocks to career decision making. CDDA seemed only partially able to discriminate…

Compares direct and indirect writing assessment strategies and contrasts them in terms of the relationship each has to specific classroom decision-making situations, the components of writing assessed, practical testing matters, characteristics of test exercises, test scoring procedures, and procedures for determining test quality. (HOD)

The project explored the assessment of vocational decision-making capacities of handicapped students within the educational system, specifically the Decision-Making Interview (DMI) instrument. The adaptability, reliability, validity, and utility of the DMI with handicapped secondary students were investigated. Under an instrument adaptation phase…

Examined utility of My Vocational Situation (MVS) in identifying career decision difficulties within population of adult psychiatric patients (N=300). Found that MVS could successfully identify those patients in need of vocational training and counseling. (Author/NB)

Poor performance of medical students, residents, and faculty on a newly developed Inventory of Cognitive Biases in Medicine, suggests that cognitive biases detract from reliance on logical and statistical strategies in medical decision making. The test shows promise for use in instructional and research efforts to reduce such bias. (Author/MSE)

A scale was developed for measuring the persuasive impact of qualitative and quantitative evaluation reports on decision makers. Using two exploratory (n=192 graduate and undergraduate students) and two confirmatory (n=200 administrators) samples, researchers developed a 28-item Likert-type scale that demonstrated high reliability and validity.…

Most seasoned test developers recognize the importance of thoughtful decision making when constructing a test. Unfortunately, many classroom achievement tests are created by novice test developed who have not received sufficient instruction in item writing (G. Gulliksen, 1986; R. J. Stiggins, 1991). The result is often a test that is poorly…

The effectiveness of representative teacher evaluation instruments for assessing the eligibility of public school teachers for merit pay was evaluated via a literature review. Elementary school principals' (N=200) ratings of teachers were analyzed. Rating instruments may be useful for making initial decisions about who should be eligible for…

Although portfolio assessment is becoming increasingly popular, it may not survive unless portfolio scoring can meet the demands of large-scale assessment standards. The results of studies of interrater reliability with large-scale portfolio assessments have been mixed. This paper reports the scoring results of a nationwide portfolio pilot in…

The COIN Basic Skills and Career Interest Survey is a two-part measure using the combined scores of the Wonderlich Basic Skills Test (E. Long, V. Artese, and W. Clonts, 1994) (WBST) and the COIN Career Interest Survey (R. Durgin, J. Ryan, and R. Ryan, 1995; COIN Educational Products, Inc.) (CIS). The third part of this package involves the…

This paper discusses how to determine whether a test was developed in a legally defensible manner, reviewing general issues, specific cases bearing on different types of test use, some evaluative dimensions, and evidence of test quality. Tests constructed and used according to existing standards will generally stand legal scrutiny. (SLD)

The Career Interest Inventory is a paper-and-pencil, self-report career guidance instrument that is designed to assist students in grades 7 through 12 and adults with career decisions. It can be administered to individuals or groups and alone or in combination with the Differential Aptitude Test. It is available in machine-scored and hand-scorable…

A basic technical framework is provided for the design and use of mastery tests. The Mastery Testing Project (MTP) prepared this framework using advanced mathematics supplemented with computer simulation based on real test data collected by the South Carolina Statewide Testing Program. The MTP focused on basic technical issues encountered in using…