Fitting wrongly specified models to observed data may lead to invalid inferences about the model parameters of interest. The current study investigated the performance of the posterior predictive model checking (PPMC) approach in detecting model-data misfit of the hierarchical rater model (HRM). The HRM is a rater-mediated model that incorporates…

Computer programming is a difficult course for many students. Prior works advocated for group learning pedagogies in pursuit of higher-level reasoning and conceptual understanding. However, the methodological gaps in existing implementations warrant further research. This study conducted a three-armed cluster-randomized controlled trial to…

Since the pandemic, the United States has faced tremendous teacher shortages (Riser-Kositsky, 2021) and national organizations have called for measures to address barriers related to teacher hiring and retention (NEA, 2022a., b; White House, 2022). Many states -- including California -- require pre-service teachers to pass state performance…

These are the appendices for the technical report, "An Investigation of the Comparability of Commission-Approved Teaching Performance Assessment Models." California's Commission on Teacher Credentialing (Commission) requires all programs of preliminary multiple and single subject teacher preparation to use a Commission-approved Teaching…

Performance assessments, scenario-based tasks, and other groups of items carry a risk of violating the local item independence assumption made by unidimensional item response theory (IRT) models. Previous studies have identified negative impacts of ignoring such violations, most notably inflated reliability estimates. Still, the influence of this…

This study integrated the Spiral Curriculum approach into the Robust Learning Model as part of a continuous improvement process that was designed to improve educational effectiveness and then assessed the differences between the initial and integrated models as well as the predictability of the first course in the integrated learning model on a…

This dissertation is comprised of three papers that propose and apply psychometric models to deal with complexities and challenges in large-scale assessments, focusing on modeling rater effects and complex learning progressions. In particular, three papers investigate extensions and applications of multilevel and multidimensional item response…

Peer assessment has been increasingly integrated in educational settings as a strategy to foster student learning. Yet little has been studied about how students at different learning levels may benefit from peer assessment. This study examined how peer-assessment and students' learning levels influenced students' project performance using a…

We study whether changes in school emphasis on academic success (SEAS) and safe schools (SAFE) may explain the increased science performance in Norway between TIMSS 2007 and 2011. Two-level structural equation modelling (SEM) of merged TIMSS data was used to investigate whether changes in levels of SEAS and SAFE mediate the changes in science…

Background/Context: From two decades of research on the grading practices of teachers in secondary schools, researchers discovered that teachers evaluated students on numerous factors that do not validly assess a student's achievement level in a specific content area. These consistent findings suggested that traditional grading practices evolved…

Performance assessments rely on the expert judgment of raters for the measurement of the quality of responses, and raters unavoidably introduce error in the scoring process. Defined as the tendency of a rater to assign higher or lower ratings, on average, than those assigned by other raters, even after accounting for differences in examinee…

Many theories, methods, and practices are utilized to evaluate teachers with the intention of determining teacher effectiveness to better inform decisions about retention, tenure, certification and performance-based pay. In the 21st century there has been a renewed emphasis on teacher evaluation in public schools, largely due to federal "Race…

Policymakers appear increasingly inclined to utilize measures of student achievement, often state assessment results, to inform high-stakes teacher personnel decisions. This has been spurred on by the federal government's Teacher Incentive Fund (TIF) and Race to the Top (RttT) grant programs, each of which urge states and localities to tie teacher…

The present study examined the long-term usefulness of estimated parameters used to adjust the scores from a performance assessment to account for differences in rater stringency. Ratings from four components of the USMLE[R] Step 2 Clinical Skills Examination data were analyzed. A generalizability-theory framework was used to examine the extent to…

Differences in intern performance, as measured by a Praxis III-similar instrument were found between interns supervised in three supervisory models: Traditional triad model, cohort model, and distance supervision. Candidates in this study's particular form of distance supervision were not as effective as teachers as candidates in traditional-triad…