The efficiency of cognitive component skills is typically assessed with speeded performance tests. Interpreting only effective ability or effective speed as efficiency may be challenging because of the within-person dependency between both variables (speed-ability tradeoff, SAT). The present study measures efficiency as effective ability…

Researchers have documented the impact of rater effects, or raters' tendencies to give different ratings than would be expected given examinee achievement levels, in performance assessments. However, the degree to which rater effects influence person fit, or the reasonableness of test-takers' achievement estimates given their response patterns,…

The array of different philosophical positions underlying contemporary views on competence, assessment strategies and justification have led to advances in assessment science. Challenges may arise when these philosophical positions are not considered in assessment design. These can include (a) a logical incompatibility leading to varied or…

Performance assessments (PAs) offer a more authentic measure of higher order skills, which is ideal for competency-based education (CBE) especially for students already in the workplace and striving to advance their careers. The goal of the current study was to examine the validity of undergraduate PA score interpretation in the college of IT at a…

We investigated the ecological validity of performance measures from a computer-based assessment tool that utilises scripted video vignettes. The intended purpose of this tool is to assess the maintenance and repair skills of automotive technician apprentices, complementing traditional hands-on assessment formats from the German journeymen's…

Background: A holistic approach to performance assessment recognizes the theoretical complexity of multifaceted critical thinking (CT), a key objective of higher education. However, issues related to reliability, interpretation, and use arise with this approach. Aims and Method: Therefore, we take an analytic approach to scoring students' written…

This paper introduces the new Massachusetts Performance Assessment for Leaders (PAL) and uses critical policy analysis to re-examine the validity evidence (using the 2014 Standards for Educational and Psychological Testing and a theory of multicultural validity) for the use and interpretation of the PAL in regards to emerging school leadership.…

This study utilizes an argument-based approach to validation to examine the implications of reliability in order to further differentiate the concepts of score and decision consistency. In a methodological example, the framework of generalizability theory was used to estimate appropriate indices of score consistency and evaluations of the…

Standardized measures are often used as an index of students' reading comprehension and scores have important implications, particularly for students who perform below expectations. This study examined secondary-level students' patterns of responding and the prevalence and impact of non-attempted items on a timed, group-administered,…

The measurement of higher-order competencies within a tertiary education system across countries presents methodological challenges due to differences in educational systems, socio-economic factors, and perceptions as to which constructs should be assessed (Blömeke, Zlatkin-Troitschanskaia, Kuhn, & Fege, 2013). According to Hart Research…

At the University of California, Los Angeles, the Test of Oral Proficiency (TOP), an internally developed oral proficiency test, is administered to international teaching assistant (ITA) candidates to ensure an appropriate level of academic oral English proficiency. Test taker performances are rated live by two raters according to four subscales.…

The 2014 Michigan Public High School Context and Performance Report Card is the Mackinac Center's second effort to measure high school performance. The first high school assessment was published in 2012, followed by the Center's 2013 elementary and middle school report card, which used a similar methodology to evaluate school performance. The…

The purpose of this article is to provide an overview of the process used to examine the inter-rater reliability of the Teacher Work Sample (TWS) Scoring Rubric involved with the senior culminating experience for teacher candidates used at a large comprehensive university. The study compared holistic and analytic scores reported by Student Teacher…

Assessing a high school's effectiveness is not straightforward. Comparing a school's standardized test scores to those of other schools is one approach to measuring effectiveness, but a major objection to this method is that students' test scores tend to be related to students' "socioeconomic" status--family household income, for…

Much of the argument about comparability of examination standards is at cross-purposes; contradictory positions are in fact often both defensible, but they are using the same words to mean different things. To clarify this, two broad conceptualisations of standards can be identified. One sees the standard in the observed phenomena of performance…