Compares the costs of four assessment formats: (1) multiple choice; (2) open ended; (3) laboratory station; and (4) full investigation. Tracks the amount of time spent preparing the devices, developing scoring consistency for the devices, and scoring the devices as they were developed. Compares times as if 1,000 students completed each assessment.…

Studied the interrater reliability of a portfolio assessment used in a small-scale program evaluation. Investigated analytic, combined analytic, and holistic family literacy portfolios from an Even Start program. Results show that at least three raters are needed to obtain acceptable levels of reliability for holistic and individual analytic…

Determined the robustness of several types of scoring (number-right; one-, two-, and three-parameter item response; finite-state, and partial-credit) with respect to the violation of the equally classifiable options and option independence made in finite-state scoring using analysis of test responses of 1,232 high school seniors. (SLD)

Discusses the design of online scoring software for computer-based education supporting high psychometric and fairness standards. Describes a system from the point of view of graders and supervisors; the design of the underlying database; data integrity and confidentiality; and the advantages of database design and online grading. (PEN)

Describes how San Diego State University created a collaborative, objective, and quantitative metric, or rubric, to support its decision-making process regarding a Web portal. Discusses how the rubric was used to score different portal software systems during demonstrations and as a checklist and inventory to describe potential options. (EV)

P.A.C.E.S. stands for Participation, Appearance, Cleanup, Engineering, and Safety. The author has traditionally used design briefs to set the limits on processes and materials to solve a given problem. The design brief brings out all kinds of "out of the box" thinking, with many correct answers to solve the problem. The P.A.C.E.S. rubric ties the…

Teenagers' innate interest with the justice system is one of the reasons that so many high school literary classics teem with criminals, controversial issues, and trials. Novels such as "To Kill a Mockingbird," "A Separate Peace," "The Crucible," and "Twelve Angry Men" feature high-impact trials. In the author's desire to tap into this interest,…

Classroom assessment information should be the basis for important classroom processes and outcomes: students' study and work patterns, students' understanding of what they are learning, and teachers' instructional and grading decisions. Attention to principles of assessment quality, especially validity and reliability, increases confidence in the…

When raters assign different scores to a performance task, a method for resolving rating differences is required to report a single score to the examinee. Recent studies indicate that decisions about examinees, such as pass/fail decisions, differ across resolution methods. Previous studies also investigated the interrater reliability of…

This article introduces the concept of Personal Educational Tools (PETs) and places these tools within the context of existing rationales for using technology for teaching, learning, and instruction. An identification of the distinguishing characteristics of these devices is followed by the conjecture that these types of classifying…

The act of collaboration is important when improving communication between administration and teachers, guiding school personnel to collectively clarify teaching and learning outcomes, and facilitating school stakeholders to engage in strategic institutional planning. As necessary as collaboration is, adults often do not know how to collaborate…

Instructional rubrics can help students to improve their writing skills and become self-regulated writers. Clear, accessible instructional rubrics give students repeated practice with planning, revising, and editing. It is also noted that using rubrics for self-assessment and peer assessment will help the students navigate the writing process in…

The extent to which the Comprehensive System for the Rorschach is reliably scored has been a topic of some controversy. Although several studies have concluded it can be scored reliably in research settings, little is known about its reliability in field settings. This study evaluated the reliability of both response-level codes and protocol-level…

Many studies have indicated that at least 2 raters should score writing assessments to improve interrater reliability. However, even for assessments that characteristically demonstrate high levels of rater agreement, 2 raters of the same essay can occasionally report different, or discrepant, scores. If a single score, typically referred to as an…

This article proposes a practical method that capitalizes on the availability of information from multiple tests measuring correlated abilities given in a single test administration. By simultaneously estimating different abilities with the use of a hierarchical Bayesian framework, more precise estimates for each ability dimension are obtained.…