The consensus-based assessment has long been a prevalent methodology employed in "panel crit" settings in design education and professional design awards. Acknowledging the subjective nature of design evaluation and its importance to the design development process, we report two studies investigating the effect of three levels of design…

In this column, the author revisits who we are as assessment professionals, how we are trained, what our role is, and what our responsibilities are--to ourselves as a community, to our colleagues in education, at our institution, and to those we serve-- especially from an equity lens. They also share a few strategies to address the gaps they see.

Learning to interpret data in context is an important educational outcome. To assess students' attainment of this outcome, it is necessary to examine the interplay between their contextual and statistical reasoning. We describe a research method designed to do so. The method draws upon Toulmin's (1958, 2003) model of argumentation for the first…

Validation is needed for any newly developed model or framework because it requires several real-life applications. The investment made into e-learning in medical education is daunting, as is the expectation for a positive return on investment. The medical education domain requires data-wise implementation of e-learning as the debate continues…

This study examined the relationship between pre-service teachers' epistemological views and their assessment conceptions, as well as how epistemological beliefs and their components contribute to the variation in their assessment conceptions. Utilizing a quantitative descriptive correlational research design, data were collected from 197…

The assessment of media literacy is a complex task, which might attempt to reconcile a research field traditionally developed within a critical paradigm with the task of evaluating and quantifying media literacy competences through essentially quantitative methods. Despite the lack of consensus regarding how to evaluate and measure media literacy,…

This experimental study explores how source use features impact raters' judgment of argumentation in a second language (L2) integrated writing test. One hundred four experienced and novice raters were recruited to complete a rating task that simulated the scoring assignment of a local English Placement Test (EPT). Sixty written responses were…

For over 30 years, calls have been issued for the western evaluation field to address implicit bias in its theory and practice. Although many in the field encourage evaluators to be culturally competent, ontological competence remains unaddressed. Grounded in an institutionalized distrust of non-western perspectives of reality and knowledge…

Comparative judgment is gaining popularity as an assessment tool, including for high-stakes testing purposes, despite relatively little research on the use of the technique. Advocates claim two main rationales for its use: that comparative judgment is valid because humans are better at comparative than absolute judgment, and because it distils the…

Evidence of the internal consistency of standard-setting judgments is a critical part of the validity argument for tests used to make classification decisions. The bookmark standard-setting procedure is a popular approach to establishing performance standards, but there is relatively little research that reflects on the internal consistency of the…

The purpose of this dissertation is to describe a doctoral research study that comprehensively investigated school psychologists' record review practices. Record reviews are part of the psychoeducational evaluation process, and there is relatively little previous research literature on this topic. The current study aimed to investigate the…

Despite the importance of child assessment data in early childhood education, teachers struggle with its use in their work. This study uses an implementation science approach to explore facilitators and barriers to teacher use of child assessment practices in three Head Start centers and tests a specific strategy to improve teacher practice. This…

Self-regulated learning (SRL) is crucial for successful lifelong learning and an important educational goal. For students to develop SRL skills, they need appropriate SRL support from teachers in the classroom. Teachers, who are aware of their students' strengths and weaknesses in SRL, can promote SRL more adaptively. This requires teachers to…

Human judgement is widely used in workplace-based assessment despite criticism that it does not meet standards of objectivity. There is an ongoing push within the literature to better embrace subjective human judgement in assessment not as a 'problem' to be corrected psychometrically but as legitimate perceptions of performance. Taking a step back…

Scoring procedures for the constructed-response (CR) items in large-scale mixed-format educational assessments often involve checks for rater agreement or rater reliability. Although these analyses are important, researchers have documented rater effects that persist despite rater training and that are not always detected in rater agreement and…