This paper presents a multidimensional model of variation in writing quality, register, and genre in student essays, trained and tested via confirmatory factor analysis of 1.37 million essay submissions to ETS' digital writing service, Criterion®. The model was also validated with several other corpora, which indicated that it provides a…

We report on the development of a rubric to reliably qualify and score responses to the "Views of Nature of Science Questionnaire" (VNOS): The "VNOS Analysis and Scoring Rubric" (VAScoR). The "VAScoR" is designed to (a) provide systematic guidance for the qualitative analysis, and score assignment to nuanced…

The purpose of study is to examine the reliability of analytical rubrics and checklists developed for the assessment of story writing skills by means of generalizability theory. The study group consisted of 52 students attending the 5th grade at primary school and 20 raters in Mersin University. The G study was carried out with the fully crossed…

Extension awards recognize programs that have achieved outstanding accomplishments, results, and impacts in addressing contemporary issues. Whether an individual or a team is recognized, the awardees gain inspiration and motivation and may attain promotion and/or tenure. The Western Region Program Leadership Committee oversees a special award…

An approach to scoring tests with binary items, referred to as D-scoring method, was previously developed as a classical analog to basic models in item response theory (IRT) for binary items. As some tests include polytomous items, this study offers an approach to D-scoring of such items and parallels the results with those obtained under the…

Purpose: This study compared the inter- and intrarater reliability of the percentage of consonants correct (PCC) metrics and the probe scoring system between an experienced and a less experienced rater and between two experienced raters. In addition, these outcome measures' ability to reflect changes following speech intervention was measured.…

Metabolic systems form the very foundation of life and as such are broadly taught in the molecular life sciences. Here, we describe the biochemistry educator community-based development and use of an assessment instrument designed to evaluate students' ideas about metabolic pathway dynamics and regulation in undergraduate biochemistry courses.…

Item response theory "dual" models (DMs) in which both items and individuals are viewed as sources of differential measurement error so far have been proposed only for unidimensional measures. This article proposes two multidimensional extensions of existing DMs: the M-DTCRM (dual Thurstonian continuous response model), intended for…

The core concept of genetic information flow was identified in recent calls to improve undergraduate biology education. Previous work shows that students have difficulty differentiating between the three processes of the Central Dogma (CD; replication, transcription, and translation). We built upon this work by developing and applying an analytic…

Writing rubrics have been used in doctoral dissertations in Turkey to assess student writing for nearly twenty years. This study aims to determine which features are assessed in the rubrics used in doctoral dissertations. Twenty-five rubrics were selected to determine the analysis of validity and reliability, rubric dimensions, features of…

Background and Context: Educators make consequential curricular decisions, often with little support, particularly as it relates to equity and how to support all students. Objective: This paper investigates the use of a rubric to support educators evaluating computer science curricula, especially with regards to equity. Method: Seventeen…

Most teacher education assessments are criticized for lacking validity and reliability. This study describes the process of developing the Observation of Field Performance rubric to assess initial teacher candidates' classroom performance and establishing the content validity as well as reliability of the rubric. A panel of content area experts…

Students are generally required to demonstrate diverse skills when writing their bachelor thesis. Accordingly, examiners are expected to consider all these skills when assessing the thesis, regularly with one overall grade. In this study, we examine which criteria of a rubric contribute most to the overall assessment. The study is performed…

Advancements in technology and computer-based testing has allowed for greater flexibility in assessing examinee knowledge on large-scale, high-stakes assessments. Through computer-based delivery, cognitive ability and skills can be effectively assessed cost-efficiently and measure domains that are difficult or even impossible to measure with…

Recent research on rater effects in performance assessments has increasingly focused on rater centrality, the tendency to assign scores clustering around the rating scale's middle categories. In the present paper, we adopted Jin and Wang's (2018) extended facets modeling approach and constructed a centrality continuum, ranging from raters…