This article presents an empirical evaluation of the validity and reliability of a peer-assessment activity to improve academic writing competences. Specifically, we explored a large group of psychology undergraduate students with different initial writing skills. Participants (n = 365) produced two different essays, which were evaluated by their…

In this brief review, we reflect upon the key contributions of research examining children's eyewitness testimony. Children's testimonial ability became a focus of interest for researchers about 40 years ago in the wake of several high-profile child abuse cases that prompted questions about children's reliability in the face of problematic…

When numerous treatments exist for a disease (Treatments 1, 2, 3, etc), network meta-regression (NMR) examines whether each relative treatment effect (eg, mean difference for 2 vs 1, 3 vs 1, and 3 vs 2) differs according to a covariate (eg, disease severity). Two consistency assumptions underlie NMR: consistency of the treatment effects at the…

Learning in natural environments is often characterized by a degree of inconsistency from an input. These inconsistencies occur, for example, when learning from more than one source, or when the presence of environmental noise distorts incoming information; as a result, the task faced by the learner becomes ambiguous. In this study, we investigate…

Background: Occupational voice problems constitute a serious public health issue with substantial financial and human consequences for society. Modern mobile technologies such as smartphones have the potential to enhance approaches to prevention and management of voice problems. This paper addresses an important aspect of smartphone-assisted voice…

Method: Sixty English-speaking children, 7-14 years old with normal hearing, had a single study visit during which each test was administered twice. Changes on retest were summarized by within-subject standard deviation ( S[subscript w]), compared among tests, and compared with binomial model predictions. Correlates of variance were explored.…

This note confronts the common use of a single coefficient alpha as an index informing about reliability of a multicomponent measurement instrument in a heterogeneous population. Two or more alpha coefficients could instead be meaningfully associated with a given instrument in finite mixture settings, and this may be increasingly more likely the…

We examined the reliability of grading master's theses at a New Zealand university, where a variant of the academic journal review system is employed. The overall correlation between the grades recommended by internal and external markers of master's theses in psychology and applied psychology at this university was 0.39, which is similar to that…

Knee flexor and extensor muscular assessment via isokinetic dynamometry is common practice and established in the research literature. However, reporting assessment methodology regarding reciprocal and nonreciprocal movements is often vague or absent. Such methodological issues are crucial for accurate assessments. Therefore, knee extensor and…

Comparative Judgement (CJ) is an increasingly widely investigated method in assessment for creating a scale, for example of the quality of essays. One area that has attracted attention in CJ studies is the optimisation of the selection of pairs of objects for judgement. One approach is known as adaptive comparative judgement (ACJ). It has been…

This note discusses the merits of coefficient alpha and their conditions in light of recent critical publications that miss out on significant research findings over the past several decades. That earlier research has demonstrated the empirical relevance and utility of coefficient alpha under certain empirical circumstances. The article highlights…

Peer assessment is a promising solution for scaling up the grading of a large number of submissions. The reliability of evaluations is one of the critical issues in peer assessment; several probabilistic models have been proposed for obtaining reliable grades from peers. Peer correction is a similar framework, in which students are instructed to…

Trend scoring is often used in large-scale assessments to monitor for rater drift when the same constructed response items are administered in multiple test administrations. In trend scoring, a set of responses from Time "A" are rescored by raters at Time "B." The purpose of this study is to examine the ability of…

Reliability is the consistency of a set of scores that are designed to measure the same thing. Reliability is a statistical property of scores that must be demonstrated rather than assumed.

This article documents the adaptation, piloting and validation of a measure of teachers' ethical sensitivity. To create the test, we modified a measure from dentistry drawing on literature in teacher professional ethics and drew on the expertise of professional ethics scholars and practitioners. Based on the results of Rasch analysis combined with…