This note confronts the common use of a single coefficient alpha as an index informing about reliability of a multicomponent measurement instrument in a heterogeneous population. Two or more alpha coefficients could instead be meaningfully associated with a given instrument in finite mixture settings, and this may be increasingly more likely the…

We examined the reliability of grading master's theses at a New Zealand university, where a variant of the academic journal review system is employed. The overall correlation between the grades recommended by internal and external markers of master's theses in psychology and applied psychology at this university was 0.39, which is similar to that…

Knee flexor and extensor muscular assessment via isokinetic dynamometry is common practice and established in the research literature. However, reporting assessment methodology regarding reciprocal and nonreciprocal movements is often vague or absent. Such methodological issues are crucial for accurate assessments. Therefore, knee extensor and…

Comparative Judgement (CJ) is an increasingly widely investigated method in assessment for creating a scale, for example of the quality of essays. One area that has attracted attention in CJ studies is the optimisation of the selection of pairs of objects for judgement. One approach is known as adaptive comparative judgement (ACJ). It has been…

This note discusses the merits of coefficient alpha and their conditions in light of recent critical publications that miss out on significant research findings over the past several decades. That earlier research has demonstrated the empirical relevance and utility of coefficient alpha under certain empirical circumstances. The article highlights…

Peer assessment is a promising solution for scaling up the grading of a large number of submissions. The reliability of evaluations is one of the critical issues in peer assessment; several probabilistic models have been proposed for obtaining reliable grades from peers. Peer correction is a similar framework, in which students are instructed to…

Trend scoring is often used in large-scale assessments to monitor for rater drift when the same constructed response items are administered in multiple test administrations. In trend scoring, a set of responses from Time "A" are rescored by raters at Time "B." The purpose of this study is to examine the ability of…

Reliability is the consistency of a set of scores that are designed to measure the same thing. Reliability is a statistical property of scores that must be demonstrated rather than assumed.

This article documents the adaptation, piloting and validation of a measure of teachers' ethical sensitivity. To create the test, we modified a measure from dentistry drawing on literature in teacher professional ethics and drew on the expertise of professional ethics scholars and practitioners. Based on the results of Rasch analysis combined with…

There is a clear need for highly competent health sciences experts. No instrument currently exists for assessing the generic competences of health sciences students. The aim of this study is to develop and psychometrically test the Health sciences Generic Competence (HealthGenericCom) instrument. The instrument development four step process has…

Despite several psychometric advantages over the 50-item Autism Spectrum Quotient, an instrument used to measure autistic traits, the abridged AQ-28 and its cross-cultural validity have not been examined as extensively. Therefore, this study aimed to examine the factor structure and measurement invariance of the AQ-28 in 818 Dutch (M[subscript…

Background: As delays or disorders in early language and communication are the most prevalent symptom in children with disabilities, early screening is crucial to promote prevention, early diagnosis, and intervention. However, to the best of our knowledge, no screening tool is available for the joint assessment of early language and social…

Since its release in late 2022, ChatGPT and subsequent generative artificial intelligence (GAI) tools have raised a wide variety of questions and concerns for the field of technical communication: How will these tools be incorporated into professional settings? How might we appropriately integrate these tools into our research and teaching? In…

This article reports on the development and validation of the Instrumental Practice Goal Realization Inventory (IPGRI) designed to assess the process of self-directed study, beginning with setting the intention to practice and ending with the evaluation of one's performance. This new tool is based on the theoretical model of action phases. The…

This review is aimed at identifying assessment instruments used to measure treatment outcomes in children with autism spectrum disorder who received early and intensive behavioral interventions. Forty three articles were included and appraised using the Council for Exceptional Children's Standards for Evidence Based Practice quality index rater.…