This study aimed to develop a rubric for evaluating digital contents specifically Web 2.0 tools in education to improve the quality of digital content, as well as to assess the rubric's validity and reliability. Data were collected from pre-service science teachers in their third year. In the development process, expert opinions were sought, and…

The students' dominant language might influence how they use and process a rubric and its subsequent effect on task performance. However, our knowledge about these effects is limited. This study investigates how the dominant language of students is associated with their rubric reading patterns and their task performance in a written landscape…

Recent developments in the field of artificial intelligence allow for improved performance in the automated assessment of extended response items in mathematics, potentially allowing for the scoring of these items cheaply and at scale. This study details the grand prize-winning approach to developing large language models (LLMs) to automatically…

The extensive use of automated speech scoring in large-scale speaking assessment can be revolutionary not only to test design and rating, but also to the learning and instruction of speaking based on how students and teachers perceive and react to this technology. However, its washback remained underexplored. This mixed-method study aimed to…

While automated essay scoring (AES) can reliably grade essays at scale, automated writing evaluation (AWE) additionally provides formative feedback to guide essay revision. However, a neural AES typically does not provide useful feature representations for supporting AWE. This paper presents a method for linking AWE and neural AES, by extracting…

Advances in computational language models increasingly enable adaptive support for self-regulated learning (SRL) in digital learning environments (DLEs; eg, via automated feedback). However, the accuracy of those models is a common concern for educational stakeholders (eg, policymakers, researchers, teachers and learners themselves). We compared…

Evaluation is an essential component of the learning process when discerning learning situations. Assessing natural language responses, like short answers, takes time and effort. Artificial intelligence and natural language processing advancements have led to more studies on automatically grading short answers. In this review, we systematically…

Reflective Learning Journals are used to engage students in the learning process. To motivate students to complete the Reflective Learning Journal it is common practice to allocate marks and it is in this area that problems arise. To address the problems rubrics have been used however, most rubrics are based on assigning marks to very…

To ensure each county board of education provides a thorough and efficient education for its students, the West Virginia Board of Education (WVBE) annually reviews county scorecards and efficiency profiles provided by the Office of Accountability. County statuses are reviewed and determined based on multiple measures of student performance and…

This paper examined the ability of students to pose a problem in a free posing situation about sets in terms of the problem text, compatibility with mathematics principles, the problem type and structure, solvability, and mathematical complexity. Additionally, students were probed to determine the difficulties they encountered during the problem…

The development of a set of rubrics for teachers and curriculum designers interested in the quality of numeracy tasks emerged from our struggle with an attempt to clarify the distinction between numeracy and mathematics. Shifting our perspective from asking what numeracy and mathematics are to asking what they are about led to the…

Developing program-level assessment systems creates an opportunity for faculty to think deeply about their student learning objectives while creating structures that provide meaningful and actionable feedback. Unfortunately, many programs lack the expertise and/or support at the programmatic or institutional level to create efficient and effective…

Student feedback is crucial for evaluating the performance of teachers and the quality of teaching. Free-form text comments obtained from open-ended questions are seldom analyzed comprehensively since it is difficult to interpret and score compared to standardized rating scales. To solve this problem, the present study employed aspect-level…

In a targeted double-scoring procedure for performance assessments that are used for licensure and certification purposes, a subset of responses receives an independent second rating if the first rating falls into a preidentified critical score range (CSR) where an additional rating would lead to considerably more reliable pass-fail decisions.…

Recent research has provided evidence that performance change during a student's test event can indicate the presence of test-taking disengagement. Meaningful performance change implies that some portions of the test event reflect assumed maximum performance better than others and, because disengagement tends to diminish performance,…