Automated scoring for divergent thinking (DT) seeks to overcome a key obstacle to creativity measurement: the effort, cost, and reliability of scoring open-ended tests. For a common test of DT, the Alternate Uses Task (AUT), the primary automated approach casts the problem as a semantic distance between a prompt and the resulting idea in a text…

Rubrics are widely used as instructional and learning instrument. Though they have been claimed to have positive effects on students' learning, these effects have not been meta-analyzed. Our aim was to synthesize the effects of rubrics on academic performance, self-regulated learning, and self-efficacy. The moderator effect of the following…

This paper illustrates how the combination of teacher and computer guidance can strengthen collaborative revision and identifies opportunities for teacher guidance in a computer-supported collaborative learning environment. We took advantage of natural language processing tools embedded in an online, collaborative environment to automatically…

Manually grading the Response to Text Assessment (RTA) is labor intensive. Therefore, an automatic method is being developed for scoring analytical writing when the RTA is administered in large numbers of classrooms. Our long-term goal is to also use this scoring method to provide formative feedback to students and teachers about students' writing…

A neuropsychologist describes a child's performance on a measure of short-term verbal memory as falling in the low average range. Another neuropsychologist reports that a child scored in the below average range. A third neuropsychologist describes a child's performance as mildly impaired. Yet, all three are referring to the same score on the same…

Many high-stakes examinations in the United Kingdom (UK) use both constructed-response items and selected-response items. We need to evaluate the inter-rater reliability for constructed-response items that are scored by humans. While there are a variety of methods for evaluating rater consistency across ratings in the psychometric literature, we…

This brief report describes the conception, development, and use of a rubric in evaluating the feasibility of a new program. The evaluators searched for a meta-analytic tool to help organize ideas about what data to collect, and why, in order to create a detailed story of feasibility of implementation for the client. The main advantage of using…

In test-centered standard-setting methods, borderline performance can be represented by many different profiles of strengths and weaknesses. As a result, asking panelists to estimate item or test performance for a hypothetical group study of borderline examinees, or a typical borderline examinee, may be an extremely difficult task and one that can…

New Meridian Corporation has developed the "Quality Testing Standards and Criteria for Comparability Claims" (QTS). The goal of the QTS is to provide guidance to states that are interested in including content from the New Meridian item bank and intend to make comparability claims with "other assessments" that include New…

Inference is considered an important factor in comprehension models and has been described as a causal factor in predicting comprehension. To date, specific tests for inference are rare and often rely on specific thematic texts. This reliance on thematic inference may raise some concerns as inference is related to prior text-specific knowledge.…

This study aimed to propose a new method for scoring C-Tests as measures of general language proficiency. In this approach, the unit of analysis is sentences rather than gaps or passages. That is, the gaps correctly reformulated in each sentence were aggregated as sentence score, and then each sentence was entered into the analysis as a polytomous…

Emergent writing--the process where young children begin to experiment with written language--is an important contributor to the development of literacy skills. One way for teachers to support the development of writing skills in preschool-aged children is by integrating the Classroom Assessment Scoring System (CLASS) as a framework to foster…

Hispanic-Serving Institutions (HSIs) play an important role in educating Latinos. The purpose of this article is to apply and extend the Typology of Hispanic-Serving Institutions Organizational Identities framework and propose a systematic rubric as a tool for assessment and evaluation. The article contains analyses and discourse for HSIs…

In the fairy tale "Goldilocks," a young girl enters the home of three bears. As she explores the porridge, chairs, and beds, in each situation she is seeking what is "just right." It seems that Goldilocks is the perfect metaphor for describing learners experiencing universal design for learning (UDL) because it highlights the…

Assessment methods vary widely across undergraduate physiology courses. Here, a cumulative oral examination was administered in two sections of a 300-level undergraduate physiology course. Student performance was quantified via instructor grading using a rubric, and self-perceptions (n = 55) were collected via survey. Overall, students affirmed…