Background: Stakeholders' interpretations of the findings of large-scale educational assessments can influence important decisions. In the context of educational assessment, standard-setting remains an especially critical element, because it is complex and largely unstandardized. Instruments established by means of standard-setting procedures such…

Purpose: This research aimed to identify reliable tasks discriminating French-speaking adolescents with developmental language disorder (DLD) from their peers with typical language (TL) and to assess which linguistic domains represent areas of particular weakness in DLD. Unlike English, morphosyntax has not been identified as a special area of…

Research on the use of standardized test scores in higher education reveals significant variations in attitudes and perceptions of language proficiency tests among test score users. Most test score users have limited knowledge about test score interpretations in terms of what English as additional language (EAL) students typically know and can do…

The aim of this study was to compare, by simulation, the accuracy of mapping a cut-score from one test to another by expert judgement (using the Angoff method) versus the accuracy with a small-sample equating method (chained linear equating). As expected, the standard-setting method resulted in more accurate equating when we assumed a higher level…

An important consideration in standard setting is recruiting a group of panelists with different experiences and backgrounds to serve on the standard-setting panel. This study uses data from 14 different Angoff standard settings from a variety of medical imaging credentialing programs to examine whether people with different professional roles and…

Setting performance standards is a judgmental process involving human opinions and values as well as technical and empirical considerations. Although all cut score decisions are by nature somewhat arbitrary, they should not be capricious. Judges selected for standard-setting panels should have the proper qualifications to make the judgments asked…

Despite concerns about their validity, self-report surveys remain the primary data collection method in the research of self-regulated learning (SRL). To address some of these concerns, we took a data set comprised of college students' self-reported beliefs and behaviours related to SRL, assessed across three surveys, and examined it for instances…

This study examined the performance of the First Year Inventory (FYI; version 2.0), a community-normed parent-reported screening instrument, in a high-risk (HR) sample of 12-month-olds with older siblings diagnosed with autism spectrum disorder (ASD). The FYI 2.0 was completed by parents of 86 HR infants and 35 low-risk control infants at age…

Over half of community college students place into developmental education, resulting in significant financial costs. We extend previous research demonstrating that using placement tests to assign students into developmental courses results in frequent misplacement. We use Florida data to explore the extent to which students are misplaced into…

For years Georgia's HOPE Scholarship program provided full tuition scholarships to high-achieving students. State budgetary shortfalls reduced its generosity in 2011. Under the new rules, only students meeting more rigorous merit-based criteria would retain the original scholarship covering full tuition, now called the Zell Miller Scholarship,…

We examined how ACT® Aspire® (from grades 8 to 10), PreACT® (from grade 10), and ACT® test scores (from grades 10 and 11) relate to students' chances of scoring a 3 or higher as well as 4 or higher on selected Advanced Placement (AP) exams. Data for the study was available for 49,220 students from 318 high schools who had taken at least one AP…

The purpose of this study was to develop a standard-setting method appropriate for use with a diagnostic assessment that produces profiles of student mastery rather than a single raw or scale score value. The condensed mastery profile method draws from established holistic standard-setting methods to use rounds of range finding and pinpointing to…

This article provides an overview of the Hofstee standard-setting method and illustrates several situations where the Hofstee method will produce undefined cut scores. The situations where the cut scores will be undefined involve cases where the line segment derived from the Hofstee ratings does not intersect the score distribution curve based on…

Raters must score accurately and consistently for classroom observation scores to be valid. This requires (a) a standard defining when scoring is accurate and consistent enough and (b) measuring and remediating rater performance against that standard. Current practice has focused on this second problem to the exclusion of the first. My goal here…

This study examined gender differences in the psychometric properties of the Student Risk Screening Scale for Internalizing and Externalizing behaviors (SRSS-IE) using item response theory methods among a sample of 2,122 middle school students. The SRSS-IE is a screening instrument used to identify students who are potentially at risk for…