Quantitative measures of systematic differences in OSCE scoring across examiners (often termed examiner stringency) can threaten the validity of examination outcomes. Such effects are usually conceptualised and operationalised based solely on checklist/domain scores in a station, and global grades are not often used in this type of analysis. In…

The 2022-2023 Indiana Assessments Policy Manual serves as the foundation for established guidelines regarding appropriate test administration in Indiana for key stakeholders including educators and Test Coordinators. The following document contains policy guidance and contains appendices which pertain to specific aspects of test implementation,…

Intelligence tests are used in a variety of settings such as schools, clinics, and courts to assess the intellectual capacity of individuals of all ages. Intelligence tests are used to make high-stakes decisions such as special education placement, employment, eligibility for social security services, and determination of the death penalty.…

We explored three approaches to resolving or re-scoring constructed-response items in mixed-format assessments: rater agreement, person fit, and targeted double scoring (TDS). We used a simulation study to consider how the three approaches impact the psychometric properties of student achievement estimates, with an emphasis on person fit. We found…

For many practical purposes, it is often assumed that the quality of a marker is directly related to their seniority. At its extreme, the assumption is that the most senior marker (the Principal Examiner) is always right even in cases where large numbers of junior markers have a collectively different opinion regarding the mark that should be…

In April 2021, The Association for Assessment and Research in Counseling Executive Council commissioned a time-referenced task group to revise the Responsibilities of Users of Standardized Tests (RUST) Statement (3rd edition) published by the Association for Assessment in Counseling (AAC) in 2003. The task group developed a work plan to implement…

One of the challenges in scoring constructed-response (CR) items and tasks is ensuring that rater drift does not occur during or across scoring windows. Rater drift reflects changes in how raters interpret and use established scoring criteria to assign essay scores. Calibration is a process used to help control rater drift and, as such, serves as…

The instructions in this manual explain the responsibilities of school administrators for the New York State Testing Program (NYSTP) Grades 3-8 English Language Arts and Mathematics Tests. School administrators must be thoroughly familiar with the contents of the manual, and the policies and procedures must be followed as written so that testing…

This study examined the scoring errors across three widely used achievement tests (Kaufman Test of Educational Achievement--Second Edition [KTEA-2], Woodcock--Johnson Tests of Achievement--Third Edition [WJ-III], and the Wechsler Individual Achievement Test--Third Edition [WIAT-III]) by novice examiners. A total of 114 protocols were evaluated for…

The instructions in this manual explain the responsibilities of school administrators for the New York State Testing Program (NYSTP) Grades 3-8 English Language Arts and Mathematics Paper-Based Field Tests. School administrators must be thoroughly familiar with the contents of the manual, and the policies and procedures must be followed as written…

The instructions in this manual explain the responsibilities of school administrators for the New York State Testing Program (NYSTP) Grades 3-8 English Language Arts and Mathematics Field Tests, and the Elementary-level (Grade 5) and Intermediate-level (Grade 8) Science Field Tests. School administrators must be thoroughly familiar with the…

The instructions in this manual explain the responsibilities of school administrators for the New York State Testing Program (NYSTP) Grades 3-8 English Language Arts and Mathematics Tests. School administrators must be thoroughly familiar with the contents of the manual, and the policies and procedures must be followed as written so that testing…

The instructions in this manual explain the responsibilities of school administrators for the New York State Testing Program (NYSTP) Grades 3-8 English Language Arts and Mathematics Tests. School administrators must be thoroughly familiar with the contents of the manual, and the policies and procedures must be followed as written so that testing…

The instructions in this manual explain the responsibilities of school administrators for the New York State Testing Program (NYSTP) Grades 3-8 English Language Arts and Mathematics Tests. School administrators must be thoroughly familiar with the contents of the manual and the policies and procedures must be followed as written so that testing…

Protocols from 108 administrations of the Reynolds Intellectual Assessment Scales were evaluated to determine the frequency of examiner errors and their impact on the accuracy of three test composite scores, the Composite Ability Index (CIX), Verbal Ability Index (VIX), and Nonverbal Ability Index (NIX). Students committed at least one…