Classroom observation rubrics are a widely adopted tool for measuring the quality of teaching and provide stable conceptualisations of teaching quality that facilitate empirical research. Here, we present four statistical approaches for analysing data from classroom observations: Factor analysis, Rasch modelling, latent class or profile analysis,…

Rubrics are popular tools to assess students' social-emotional skills. However, little research has been devoted to investigating whether rubrics' performance level descriptions validly represent the intended skills. In this study, we addressed the validity of Self-management rubrics by examining the presence of construct-irrelevant variance in…

While self-assessment is a widely explored area in educational research, our understanding of how students assess themselves, or in other words, generate self-feedback, is quite limited. Self-assessment process has been a black box that recent research is trying to open. This study explored and integrated two data collections (secondary and higher…

Formative assessment is an important driver in supporting children's writing development. This paper describes a writing rubric designed for use by teachers to formatively assess the writing of children in Pre-K to Grade 2, how the rubric was received by teachers, and its implementation in classrooms. Writing samples from 337 children in 33…

PISA presents a new image for academic achievement, which has prompted Japanese education reforms over the past decade to innovate teaching and learning for 'PISA-style literacy'. Supported by theoretical foundations, particularly with regard to the concept of 'PISA literacy' and 'authentic assessment', these reforms have accomplished progress in…

In the business disciplines, there is a disconnection between what employers expect graduates to do and what graduates are actually able to do. Limited research has been done on the interfaces of professionals' aspirations, university instructors' enactment, and university learners' experiences. To address this well-felt gap, we conducted a series…

Writing assessment is a key feature of most education systems, yet there are limitations with traditional methods of assessing writing involving rubrics. In contrast, comparative judgement appears to overcome the reliability issues that beset the assessment of performance assessment tasks. The approach presented here extends previous work on…

Human raters and machine scoring systems potentially have complementary strengths in evaluating language ability; specifically, it has been suggested that automated systems might be used to make consistent measurements of specific linguistic phenomena, whilst humans evaluate more global aspects of performance. We report on an empirical study that…

This paper details a two-stage process for standard setting using an extended pairwise comparison method. It describes the application of a new process of establishing a set of scaled and ordered performances as a basis for exemplifying standards. In stage one, the method of pairwise comparison was used to generate data from pairwise judgements…

Accurate measurement of metacognitive knowledge in reading is important. Different instruments and scoring methods have been proposed but not systematically compared for their measurement comparability across cultures and validity. Given student data from 34 OECD countries in the Programme for International Student Assessment (PISA) in 2009, we…

This paper presents a scoping review of, firstly, how teachers use digital technologies for school-based assessment, and secondly, how these assessment-purposed digital technologies are used in teacher- and student-centred pedagogies. It draws on research about the use of assessment-purposed digital technologies in school settings, published from…

Recently, comparative judgement has been introduced as an alternative method for scoring essays. Although this method is promising in terms of obtaining reliable scores, empirical evidence concerning its validity is lacking. The current study examines implications resulting from two critical assumptions underpinning the use of comparative…

This study explores the effects of the shift to emergency remote teaching on assessment practices due to COVID-19 lockdown. A total of 936 Spanish teachers from all educational levels ranging from early childhood to university participated in this nationwide survey. Four aspects were explored: (1) changes in the use of assessment instruments (e.g.…

Recent advances in machine learning have made automated scoring of learner speech widespread, and yet validation research that provides support for applying automated scoring technology to assessment is still in its infancy. Both the educational measurement and language assessment communities have called for greater transparency in describing…

This study evaluated rater accuracy with rater-monitoring data from high stakes examinations in England. Rater accuracy was estimated with cross-classified multilevel modelling. The data included face-to-face training and monitoring of 567 raters in 110 teams, across 22 examinations, giving a total of 5500 data points. Two rater-monitoring systems…