A set of procedures is described for constructing an assessment network composed of a connected system of rater and writing task banks within the context of large-scale assessments of written composition. The calibration of the assessment tasks and the measurement of individuals are viewed as separate, although complementary, activities. The…

As developer and custodian of the General Aptitude Test Battery (GATB), the U.S. Department of Labor (USDOL) is responsible for ensuring that its use meets established professional standards. Considering the logistical problems of overseeing the use of the GATB at more than 2,000 separate sites, the USDOL has confronted a monumental task in the…

This paper proposes a systematic method of measuring subjects' free recall from printed hard news stories, based on schema theories of cognition. Citing literature that demonstrates the role of text structures and text schemas in the recall of written text, the paper suggests incorporating these processes into the assessment of recall of news. In…

The reliability and validity of the State-Trait Anxiety Inventory for Children (STAIC) was studied with 675 adolescents aged 12 to 18 recruited from clinical and community sources. The STAIC is a self-report measure that has been widely used to assess state and trait anxiety of children. It has been suggested that the child version may be more…

Work Keys provides a metric that translates skill requirements for individual jobs into levels of proficiency. It has been developed as a multifunctional program with the interactive components of Job Profiling, Instructional Support, Reporting, and Assessment. The assessment component contains applied mathematics, applied technology, teamwork,…

A study examined college students' responses to writing tasks that were created by their instructors--writing tasks that constituted an important part of the instructors' course designs and that were presented to students as an integral part of the curriculum. In all, approximately 4800 independent evaluations of 796 essays were analyzed. The…

This study investigated the intrajudge consistency of Angoff-based item performance estimates. The examination used was a certification examination in an emergency medicine specialty. Ten expert panelists rated the same 24 items twice during an operational standard setting study. Results indicate that the panelists were highly consistent, in terms…

This book offers a descriptive account of the world of the British external examiner: who they are, what they do, how they are appointed, and what problems they face. Findings are based on a questionnaire survey of and personal interviews with examiners. The external examiner is independent of the university awarding the degree and quite…

This paper reports the results of an analysis of the relationship between scorer behaviors and score variability. Thirty-six essay scorers were interviewed and asked to perform a think-aloud task as they scored 24 essays. Each comment made by a scorer was coded according to its content focus (i.e. appearance, assignment, mechanics, communication,…

Students often have difficulty in evaluating the validity of a study. A conceptually and linguistically meaningful framework for evaluating research studies is proposed that is based on the discussion of internal and external validity of T. D. Cook and D. T. Campbell (1979). The proposal includes six key dimensions, three related to internal…

The Ability Explorer (AE) is a newly developed self-report inventory of abilities that is appropriate for group or individual administration. There are machine-scorable and hand-scorable versions of the test, and there are two levels. Level 1 is for students from junior high to high school, and Level 2 is for high school students and adults.…

It is sometimes sensible to think of the fundamental unit of test construction as being larger than an individual item. This unit, dubbed the testlet, must pass muster in the same way that items do. One criterion of a good item is the absence of differential item functioning (DIF). The item must function in the same way as all important…

Among the measurement techniques receiving greater attention is the context-dependent item set or testlet. The context-dependent item set consists of a scenario and related test questions. This item format is generally believed to be able to tap higher level thinking. Unfortunately, this item form leads to inter-item dependence within item sets…

Students at grades four and five were administered a writing assessment that was developed to correspond to the California Learning Assessment System (CLAS) writing tasks at grade four. Teachers were trained to score the CLAS-like tasks according to the rubric developed by the State for CLAS. In addition, 164 students at three schools in the…

This study evaluated the validity of using an earplug so that information is presented to only one ear of children (N=34) in Alberta, Canada, with learning disabilities as a means of improving listening comprehension. Children were asked to recall stories from the Auditory Comprehension Test (ACT) immediately after hearing the story. The children…