Quantitative measures of systematic differences in OSCE scoring across examiners (often termed examiner stringency) can threaten the validity of examination outcomes. Such effects are usually conceptualised and operationalised based solely on checklist/domain scores in a station, and global grades are not often used in this type of analysis. In…

Intelligence tests are used in a variety of settings such as schools, clinics, and courts to assess the intellectual capacity of individuals of all ages. Intelligence tests are used to make high-stakes decisions such as special education placement, employment, eligibility for social security services, and determination of the death penalty.…

Administration and scoring of cognitive assessments have evolved from a paper-based platform to a digital format. Since this advancement, Pearson has created a system (Q-interactive) that allows examiners to administer the WISC-V via two iPads. However, limited research exists exploring the effects of this new method of administration when…

This study examined the scoring errors across three widely used achievement tests (Kaufman Test of Educational Achievement--Second Edition [KTEA-2], Woodcock--Johnson Tests of Achievement--Third Edition [WJ-III], and the Wechsler Individual Achievement Test--Third Edition [WIAT-III]) by novice examiners. A total of 114 protocols were evaluated for…

It is widely accepted dogma that consequential decisions are better made with multiple measures, because using but a single one is thought more likely to be laden with biases and errors that can be better controlled with a wider source of evidence for making judgments. Unfortunately, advocates of using multiple measures too rarely provide detailed…

It is particularly important that GCSE and A-level marking is valid and reliable as it affects the life chances of many young people in England. Current developments in marking technology are coinciding with potential changes in procedures to ensure valid and reliable marking. In this research the effectiveness of procedures to facilitate the…

A total of 51 Wechsler Intelligence Scale for Children, Fourth Edition (WISC-IV) protocols, administered by graduate students in training, were examined to obtain data describing the frequency of examiner errors and the impact of errors on resultant test scores. Present results were generally consistent with previous research examining graduate…

Conducted study to examine whether practitioners err in administering and scoring Wechsler Adult Intelligence Scale-Revised (WAIS-R). Obtained WAIS-R protocols from 50 randomly selected psychological folders in records of 1 school district. Found that practitioners committed errors on all 50 protocols. Errors on 27 of 50 protocols were sufficient…

Examined the extent of examiner error during administration of the Wechsler Adult Intelligence Scale (WAIS) by practicing school psychologists and school psychology students eligible for state certification as psychometrists. A number of examiner item scoring and administration errors were observed for numerous subtests. (RC)

Analyzed 56 Wechsler Intelligence Scale for Children-Revised protocols completed by 1 certified and 8 licensed practitioners to examine administration and scoring mistakes. Observed numerous mistakes (failure to record examinee responses, assigning too few or too many points to answers, inappropriate questioning, and failure to obtain correct…

Investigated specific problem caused by traditional method of teaching students to administer Wechsler Adult Intelligence Scale-Revised. Analysis of 180 protocols by 26 graduate students revealed average of 8.8 mistakes per protocol. When errors were corrected, 81 percent of Full Scale intelligence quotients were changed. Students' performance…

Investigated most frequent types of examiner errors made by graduate students (n=26) in administering Wechsler Intelligence Scale for Children-Revised (WISC-R) and examined on which items these mistakes were most likely to occur. Findings identified deficiencies in traditional methods of teaching students how to administer the WISC-R. Students…

Analyzed for examiner errors 55 Wide Range Achievement Test-Revised (WRAT-R) protocols completed by 9 practitioners for metropolitan school district. All practitioners made errors, which occurred on 95 percent of protocols and averaged 3.0 errors per protocol. Most frequent errors included failures to obtain correct ceiling or basal, and failures…

Evaluated the technical aspects of three Wechsler Intelligence Scale for Children-Revised (WISC-R) administrations of five psychology graduate students using the WISC-R Administration Observational Checklist (WAOC) to evaluate interrater agreement. Students performed significantly better on the second than on the first observation, with…

Bias effect was mainly accounted for by male experimenters testing subjects under conditions of nonautomated stimulus presentation and by female experimenters testing subjects under automated conditions. (Authors)