Comparative Judgement (CJ) is traditionally and primarily used to compare written texts. In this study we explored whether we could extend its use to comparing audio files. We used GCSE Music portfolios which contained a mix of audio recordings, musical scores and text documents. Fifteen judges completed two exercises: one comparing musical…

In this study we explored the potential of adaptive comparative judgement (ACJ) as a medium for peer assessment and for the giving and receiving of peer feedback. ACJ is a marking protocol in which the assessor (or judge) merely compares two answers and chooses a winner. Repeated judgements and a suitable sorting algorithm allow marks to be…

While portfolios are increasingly used to assess competence, the validity of such portfolio-based assessments has hitherto remained unconfirmed. The purpose of the present research is therefore to further our understanding of how assessors form judgments when interpreting the complex data included in a competency-based portfolio. Eighteen…

Adaptive Comparative Judgement (ACJ) is an alternative to conventional marking in which the assessor (judge) merely compares two answers and chooses a winner. (Scripts are typically uploaded to the CompareAssess interface as pdf files and are presented side-by-side.) Repeated comparisons and application of the sorting algorithm leads to scripts…

We investigated the use of adaptive comparative judgment to evaluate the middle school student learning, engagement, and experience with the design process in an open-ended problem assigned in a technology and engineering education course. Our results indicate that the adaptive comparative judgment tool effectively facilitated the grading of the…

In this article, we discuss conflict between law and science relative to the presumption in special education law that multidisciplinary teams and others identify the causes of problems giving rise to special education needs. First, we explain eligibility criteria, highlighting ambiguities therein and why criteria constitute a mandate for causal…

Adaptive Comparative Judgement (ACJ) is a modification of Thurstone's method of comparative judgement that exploits the power of adaptivity, but in scoring rather than testing. Professional judgement by teachers replaces the marking of tests; a judge is asked to compare the work of two students and simply to decide which of them is the better.…

In order for standard setting to retain public confidence, it will be argued there are two important requirements. One is that the judges' allocation of students to performance bands would yield results broadly consistent with the expectation of the wider educational community. Secondly, in the absence of any change in educational performance,…

The main objective of this article is to present the results of a study aimed at determining, classifying and evaluating practices of interest for general competency development and assessment in undergraduate programmes. The study encompassed the following phases: (1) focus group in order to establish a starting point regarding competency…

The consistency of assessment demands is important to validity. This research investigated the comparability of the demands of college-assessed units within a vocationally related qualification, drawing on methodological approaches that have previously been used to compare assessments. Assessment materials from five colleges were obtained. After…

Central to the standards-based assessment validation process is an examination of the alignment between state standards and test items. Several alignment analysis systems have emerged recently, but most rely on either traditional rating or matching techniques. Little, if any, analyses have been reported on the degree of consistency between the two…

A study examined how debate coaches determine the validity of a ballot and how coaches can pass those validity judgments on to students. A survey consisting of five questions was attached to judging schedules at the 1998 L.E. Norton Individual Events Tournament held at Bradley University (Illinois) in November 1998. Surveys were distributed to 64…

A study investigated the content of individual forensic events ballots. In the study, 150 high school and 274 college ballots were randomly selected from 17 tournaments, and each comment on each ballot was evaluated and coded. Data indicated that the overwhelming majority of comments on high-school ballots (64.57%) were positive comments; 32.68%…

Noting that the skill of debate is essential in a free democratic society, this paper presents a rationale for the use of judging philosophy statements at all intercollegiate debate tournaments. The paper considers: the role of the debate critic; discussion of judging philosophy statements; and advantages to be gained through the use of…

Noting that the use and utility of judging philosophy statements have long been a contested issue in intercollegiate debate, this paper examines the rationale for using philosophy statements, discusses the effectiveness of philosophy statements, and advocates the use of such statements at Pi Kappa Delta's National Tournament. The paper notes that…