This research aimed to develop and validate a rubric to assess Artificial Intelligence (AI) chatbots' effectiveness in accomplishing tasks, particularly within educational contexts. Given the rapidly growing integration of AI in various sectors, including education, a systematic and robust tool for evaluating AI chatbot performance is essential.…

Bias assessment (systematic errors) is fundamental in industry and service laboratories, where reliable results must be obtained to give correct answers to specific problems. Therefore, knowledge and practice in quality methodologies is of fundamental importance for students. Unfortunately, laboratory lessons often focus on connecting theory and…

The UK-based article develops a quantitative method for measuring 8-9-year-old children's Gender Ability Beliefs through drawings, assessing the reliability and validity of the measure and its association with respondents' self-reported gender. The measure, originally used in the US by Beilock et al. (2010), required respondents to draw two…

The main objective of this study is to revise and validate the assessment of self-presentation skills of middle school students. The assessment is based on existing self-assessment scales and adaptively modified for a more accurate assessment of middle school students' self-presentation skills. Considering the characteristics of middle school…

To address the computational constraints of parameter estimation in the polytomous Cognitive Diagnosis Model (pCDM) in large-scale high data volume situations, this study proposes two two-stage polytomous attribute estimation methods: P_max and P_linear. The effects of the two-stage methods were studied via a Monte Carlo simulation study, and the…

Integrative learning is an important outcome for graduates of higher education. Therefore, it should be well-defined and assessed reliably. The American Association of Colleges & Universities has developed a rubric to define and assess integrative learning, but it has low reliability. This pilot study examines whether this rubric's reliability…

Conventional approaches to assessment involve teachers and examiners judging the quality of learners work by reference to lists of criteria or other 'outcome' statements. This paper explores a quite different method of assessment using 'Adaptive Comparative Judgement' (ACJ) that was developed within a research project at Goldsmiths University of…

The purpose of this article is to review the literature on the issue of grading as a method and technique of expressing students' performance in terms of school reality. Initially, a growing concern about the role of assessment of student's performance in the learning and, generally, in the educational process, is highlighted. Subsequently, the…

The present study explores two rating methods for peer assessment (analytical rating using criteria and comparative judgement) in light of concurrent validity, reliability and insufficient diagnosticity (i.e. the degree to which substandard work is recognised by the peer raters). During a second-year undergraduate course, students wrote a one-page…

Complex competences, such as managing students' aggressive behaviour, are challenging to develop during teacher training. Recently, video-based simulations have been considered promising, yet suitable assessment instruments are limitedly available. This paper reports on the design and evaluation of a video-based assessment tool tailored to measure…

Although there exists an extensive amount of research on subscores and their properties, limited research has been conducted on categorical subscores and their interpretations. In this paper, we focus on the claim of Feinberg and von Davier that categorical subscores are useful for remediation and instructional purposes. We investigate this claim…

Autistic people are more likely to have co-occurring mental health conditions compared to the general population, and mental health interventions have been identified as a top research priority by autistic people and the wider autism community. Autistic adults have also communicated that quality of life is the outcome that matters most to them in…

Sleep problems are common in Rett syndrome and other neurogenetic syndromes. Actigraphy is a cost-effective, objective method for measuring sleep. Current guidelines require caregiver-reported bed and wake times to facilitate actigraphy data scoring. The current study examined missingness and consistency of caregiver-reported bed and wake times…

A strong predictor of children's language is performance on non-word repetition (NWR) tasks. However, the basis of this relationship remains unknown. Some suggest that NWR tasks measure phonological working memory, which then affects language growth. Others argue that children's knowledge of language/language experience affects NWR performance. A…

The Performance Assessment and Evaluation System (PAES) is used by all major universities in the state of Utah to measure the effective teaching skills of preservice candidates as they progress through their teaching preparation program. The resulting ratings are used to make high-stakes decisions relating to course completion as well as…