Debates about appropriate, fair and effective ways of assessing research and researchers have raged through the scientific community for decades, recently mostly concerned with discussing the merits and limitations of metric-based, quantitative assessments versus peer review-based, qualitative alternatives. Ample attention has been paid to formal…

Achieving a PhD degree is viewed by academic institutions as a major landmark of success and achievement. It gives recognition to researchers and provides entry into academia. A PhD degree is not awarded lightly and doctoral candidates undergo a rigorous examination process. This study seeks to gain a better understanding of the way in which…

This study examined the perspectives of accreditation liaison officers, institutional accreditor peer-reviewers, and chief library officer perspectives on the role that the academic librarian, academic library, and information literacy should play in institutional accreditation. The study was framed using a conceptual model based on New…

It is widely acknowledged in research that common criteria and aligned standards do not result in consistent assessment of such a complex performance as the final undergraduate thesis. Assessment is determined by examiners' understanding of rubrics and their views on thesis quality. There is still a gap in the research literature about how…

This experimental study explores how source use features impact raters' judgment of argumentation in a second language (L2) integrated writing test. One hundred four experienced and novice raters were recruited to complete a rating task that simulated the scoring assignment of a local English Placement Test (EPT). Sixty written responses were…

Innovation capability has become a necessary requirement for qualified teachers in the context of informatization. However, the validity and objectivity of existing assessments are unclear. Therefore, this study selected nine researchers to evaluate the instructional innovation capabilities of 60 pre-service teachers from a Chinese normal…

Prior research has evaluated the reliability and validity of structured visual inspection (SVI) criteria for interpreting functional analysis (FA) outcomes (Hagopian et al., 1997; Roane et al., 2013). We adapted these criteria to meet the unique needs of interpreting latency-based FA outcomes and examined the reliability and validity of applying…

Rater comments tend to be qualitatively analyzed to indicate raters' application of rating scales. This study applied natural language processing (NLP) techniques to quantify meaningful, behavioral information from a corpus of rater comments and triangulated that information with a many-facet Rasch measurement (MFRM) analysis of rater scores. The…

In this article, the authors share their reflections on how different experiences and paradigms have broadened their understanding of the work of assessment in higher education. As they collaborated to create a panel for the 2024 International Conference on Assessing Quality in Higher Education, they recognized that they, as assessment…

Evaluating the impact of Academic Literacies teaching (Lea and Street [1998. "Student Writing in Higher Education: An Academic Literacies Approach." "Studies in Higher Education" 23 (2): 157-72. doi:10.1080/03075079812331380364]) is difficult, as it involves gauging whether writers: (1) gain better understanding of what…

Recent research on rater effects in performance assessments has increasingly focused on rater centrality, the tendency to assign scores clustering around the rating scale's middle categories. In the present paper, we adopted Jin and Wang's (2018) extended facets modeling approach and constructed a centrality continuum, ranging from raters…

This study investigates the validity and reliability of generative large language models (LLMs), specifically ChatGPT and Google's Bard, in grading student essays in higher education based on an analytical grading rubric. A total of 15 experienced English as a foreign language (EFL) instructors and two LLMs were asked to evaluate three student…

Examiners' judgements play a critical role in competency-based assessments such as objective structured clinical examinations (OSCEs). The standardised nature of OSCEs and their alignment with regulatory accountability assure their wide use as high-stakes assessment in medical education. Research into examiner behaviours has predominantly explored…

The recent Next Generation Science Standards recommend students have increased opportunities engaging in modeling work scientists do. Science classroom tasks include simple inquiry compared to more complex authentic inquiry that closely resembles authentic science. Science fair participation presents students with the opportunity to engage in…

This essay provides advice for effectively reviewing conference proposals, including how to write comments that are helpful to proposal authors, how to use the "Comments to Program Chair" box, and issues to consider when assigning proposal ratings and recommending acceptance or rejection. Several benefits of reviewing proposals are…