The claim has been made for decades that college teaching, like research, should be peer reviewed if we are to see it as a serious scholarly activity; yet peer review as a method of evaluating teaching is not widespread. Interest in peer review is increasing, however, as institutions have sought ways of evaluating teaching informed by professional…

This study compares how the generative AI (GenAI) large language model (LLM) ChatGPT performs in grading university exams compared to human teachers. Aspects investigated include consistency, large discrepancies and length of answer. Implications for higher education, including the role of teachers and ethics, are also discussed. Three…

This study investigates the development and evaluation of a Retrieval-Augmented Generation (RAG)-based statistics tutor designed to assist students with quantitative analysis methods. The RAG approach was employed to address the well-documented issue of hallucination in Large Language Models (LLMs). A computer tutor was developed that utilizes…

Recent advances in AI have opened the door for the automated scoring of open-ended math problems, which were previously much more difficult to assess at scale. However, we know that biases still remain in some of these algorithms. For example, recent research on the automated scoring of student essays has shown that certain varieties of English…

Value-Added Models (VAMs) are both common and controversial in education policy and accountability research. While the sensitivity of VAMs to model specification and covariate selection is well documented, the extent to which test scoring methods (e.g., mean scores vs. IRT-based scores) may affect VA estimates is less studied. We examine the…

The integration of advanced learning analytics and data-mining technology into higher education has brought various opportunities and challenges, particularly in enhancing students' self-regulated learning (SRL) skills. Analyzing developed features for SRL support, it has become evident that SRL support is not a binary concept but rather a…

Despite growing research interest in digital multimodal composing (DMC) in L2 contexts, its developmental trajectory remains largely underexplored. To address this gap, we conducted a scientometric review using CiteSpace on 128 publications indexed in the Web of Science Core Collection. Our analysis reveals a rapid growth in DMC research over the…

This study investigates the ability of future primary school teachers to design experiments when confronted with everyday life problems, particularly those involving complex phenomena or counterintuitive outcomes. Data were collected using a content analysis approach from the written assignments submitted during a Science Education university…

This study investigates the reliability and consistency of a custom GPT-based scoring system in comparison to trained human raters, focusing on B1-level opinion paragraphs written by English preparatory students. Addressing the limited evidence on how AI scoring systems align with human evaluations in foreign language contexts, the study provides…

Valid rubrics facilitate assessing the level of complexity in students' open-ended responses. To design a valid rubric, it is essential to thoroughly define the types of responses that represent evidence of varying complexity levels. Formulating such evidence statements can be approached deductively by adopting predefined criteria from the…

Open access to novel AI tools offers unprecedented opportunities for human-AI collaboration in writing instruction and assessment. While research on using generative AI tools like ChatGPT in these contexts is emerging, more is needed to understand their effectiveness as Automated Writing Evaluation (AWE) tools. This study explores the potential of…

In measurement and evaluation processes, natural language responses are often avoided due to time, workload, and reliability concerns. However, the increasing popularity of automatic short-answer grading studies for natural language responses means such answers can now be measured more quickly and reliably. This study aims to build models for…

In signal detection rater models for constructed response (CR) scoring, it is assumed that raters discriminate equally well between different latent classes defined by the scoring rubric. An extended model that relaxes this assumption is introduced; the model recognizes that a rater may not discriminate equally well between some of the scoring…

Factor score regression has recently received growing interest as an alternative for structural equation modeling. However, many applications are left without guidance because of the focus on normally distributed outcomes in the literature. We perform a simulation study to examine how a selection of factor scoring methods compare when estimating…

Purpose: Existing research suggests that organizational learning, jointly implemented by principals and teachers may reduce agency risks and improve school management effectiveness. However, research investigating how this process occurs is lacking. The relationship between school leaders promoting the involvement of teachers in school-wide…