Objective: The objective of this study was threefold: (a) to create a rubric for response complexity (RC), defined as an admixture of response length, grammatical diversity, categorisation, and sophistication; (b) to measure behavioural and cognitive engagement through students' written responses on a school-based written activity, and (c) to…

The case study addressed in this article illustrates the value of continuous response measurement (CRM) for testing and refining messages produced for distribution to Extension audiences. We used CRM to evaluate the responses of Extension educators and Natural Resources Conservation Service technical service providers to a video describing…

Although evaluators often use an interrupted time series (ITS) design to test hypotheses about program effects, there are few empirical tests of the design's validity. We take a randomized experiment on an educational topic and compare its effects to those from a comparative ITS (CITS) design that uses the same treatment group as the experiment…

In order to evaluate the effectiveness of curricular or instructional innovations, researchers often attempt to measure change in students' conceptual understanding of the target subject matter. The measurement of change is therefore a critical endeavor. Often, this is accomplished through pre-post testing using an assessment such as a…

Community-based education programs must demonstrate effectiveness to various funding sources. The pilot study reported here (funded by CYFAR, NIFA, USDA award #2008-41520-04810) had the goal of determining if state level programs with varied curriculum could use a common evaluation tool to demonstrate efficacy. Results in parenting and youth…

Identifying students' conceptual scientific understanding is difficult if the appropriate tools are not available for educators. Concept inventories have become a popular tool to assess student understanding; however, traditionally, they are multiple choice tests. International science education standard documents advocate that assessments…

We introduce one instance of a game-based assessment designed to measure students' self-regulated learning choices. We describe our overarching measurement strategy and we present "Posterlet", an assessment game in which students design posters and learn graphic design principles from feedback. We designed "Posterlet" to assess…

Two ways of measuring change are presented and compared: A conventional "change score", defined as the difference between scores before and after an interim period, and a process-oriented approach focusing on detailed analysis of conceptually defined response patterns. The validity of the two approaches was investigated. Vocabulary…

The authors aimed to incorporate learning into the cognitive assessment framework that exists for static assessment data. In order to accomplish this, they derive a common likelihood function for dynamic models and introduce Parameter Driven Process for Change + Cognitive Diagnosis Model (PDPC + CDM), a dynamic model which tracks learning…

This report describes the value-added models used as part of teacher evaluation systems in the District of Columbia Public Schools (DCPS) and in eligible DC charter schools participating in "Race to the Top." The authors estimated: (1) teacher effectiveness in DCPS and eligible DC charter schools during the 2011-2012 school year; and (2)…

Researchers have linked self-efficacy to performance, but they have not investigated the relationship between self-efficacy and knowledge and skills tests within a training evaluation setting, which is the main purpose of this study. Additionally, researchers have acknowledged self-efficacy scores may be distorted as a result of assessors…

Directed self-guidance, whereby trainees independently practice a skill-set in a structured setting, may be an effective technique for novice training. Currently, however, most evaluation methods require an expert to be present during practice. The study aim was to determine if absolute symmetry error, a clinically important measure that can be…

Proposes a methodology for analyzing the impacts of social programs on previously unexamined subgroups. The approach estimates the impact of programs on subgroups identified by a postreatment choice while maintaining the integrity of the experimental research design. (SLD)

A difficulty associated with the use of Golub and Frederick's syntactic density score was the time required in hand tabulation. This drawback was resolved with the development by Kidder of a computer program which calculates a syntactic density score for writing samples. The purpose of this study was to examine the sensitivity of the Syntactic…