Evaluators widely agree that the involvement of key actors is a central aspect of quality practice. However, a scoping review of to what extent and how youth are included has not been explored. This is an important omission because many evaluations are done on programs that directly serve youth. We examined 159 evaluation reports in a publicly…

The purpose of this systematic review was to survey available measures of creativity for pk12 students for assessments characteristics and reporting of psychometric properties. Using the PRISMA framework, we identified 42 unique articles with 48 assessments meeting our inclusion criteria. Then, two coders independently coded all articles using a…

To address the computational constraints of parameter estimation in the polytomous Cognitive Diagnosis Model (pCDM) in large-scale high data volume situations, this study proposes two two-stage polytomous attribute estimation methods: P_max and P_linear. The effects of the two-stage methods were studied via a Monte Carlo simulation study, and the…

Since the George Floyd murder in 2020 and Black Lives Matter campaign, there has been increased interest among evaluators and PIs in Diversity, Equity and Inclusion (DEI) in their projects and evaluations. Our previous research showed a lack of clarity in understanding the distinction between the terms in practice and desire to engage with these…

Non-cognitive skills are routinely measured using self-reports in the context of large-scale international assessments. However questions remain on the adequacy of self-reports to conduct comparisons. Measures that exploit test-taker's behaviour during the completion of questionnaires or of the cognitive tests have been proposed in the literature…

We interrogate the opportunities and challenges of using mixed methods (MM) within a developmental evaluation (DE) context by drawing on two illustrative cases that investigated educational change in Canada. Methods: Multi-case design and cross-case analysis, with a focus on examining common patterns across the two cases, enabling new ways of…

Simulated testing has become more prevalent in higher education, especially as competency-based institutions begin to incorporate micro-credentials and skills certificates into their curriculum. Competency-based assessment falls outside traditional norm-based testing practices used in K-12 education, and is largely focused on criterion referenced…

Evaluators have opportunities to advance equity within evaluations, yet little research has examined whether and how evaluators center equity in evaluation practice. This paper explores whether and how evaluators in New England address inequities and advance equity throughout evaluation phases. The study uses a complementarity, sequential mixed…

In this qualitative collective case study, we investigated how participating in external longitudinal root cause analyses (RCAs) contributed to the enhancement of instructional leadership practices of the leadership teams in three turnaround elementary schools in Texas. We participated with an external team of expert professors in conducting the…

This study demonstrates the importance of Rasch Measurement Theory (RMT) in program evaluation when outcome measures need to be constructed from scratch. The paper introduces typical measure validation methods presented in program evaluation texts and discusses room for improvement. The study then illustrates how the seamless transitions from…

Research indicates that several factors variably influence teachers' approaches to classroom assessment. Building on previous assessment literacy research, this study empirically examines variability in teachers' approaches to classroom assessment across a set of dimensions including their conceptions of assessment purposes, processes, fairness,…

Characterizing students' knowledge and skill upon kindergarten entry has implications for early learning and K-12 outcomes and systems alignment. Large-scale entry assessments measure skillsets shown important for later success, so that meaningful comparisons and appropriate decisions can be made. Consequently, it is important to document the…

Preservice teachers in the field encounter materials with varying degrees of completeness and quality. Using a curriculum strategy framework that analyzes how preservice teachers read, evaluate, and adapt materials, this study compares the planning of six preservice teachers using curriculum materials of varying completeness. Preservice teachers…

The purpose of this study was to design an integrated evaluation framework that would guide the measurement of the implementation of "Empowering Modern Learners" (EML) and that would provide opportunities for a conjoint participatory evaluative format. EML is a board-wide project intended to ensure high expectations for student…

Latent profile analysis (LPA) was used to explore whether multiple measures, each designed to address one aspect of classroom quality, could collectively differentiate early childhood classrooms in both a consistent and substantively meaningful way. Using data from the Head Start Impact Study from 2002/03, Head Start and center-based early…