Objectively scoring constructed-response items on educational assessments has long been a challenge due to the use of human raters. Even well-trained raters using a rubric can inaccurately assess essays. Unfolding models measure rater's scoring accuracy by capturing the discrepancy between criterion and operational ratings by placing essays on an…

When analyzing treatment effects on test scores, researchers face many choices and competing guidance for scoring tests and modeling results. This study examines the impact of scoring choices through simulation and an empirical application. Results show that estimates from multiple methods applied to the same data will vary because two-step models…

California's Commission on Teacher Credentialing (Commission) requires all programs of preliminary multiple and single subject teacher preparation to use a Commission-approved Teaching Performance Assessment (TPA) as one of the program completion requirements for prospective teacher candidates. Three TPA models were approved by the Commission: (1)…

The literature on Automated Essay Scoring (AES) systems has provided useful validation frameworks for any assessment that includes AES scoring. Furthermore, evidence for the scoring fidelity of AES systems is accumulating. Yet questions remain when appraising the scoring performance of AES systems. These questions include: (a) which essays are…

The analysis of exemplars of different quality is a potentially powerful tool in enabling students to understand assessment expectations and appreciate academic standards. Through a systematic review methodology, this paper synthesises exemplar-based research designs, exemplar implementation and the educational effects of exemplars. The review of…

In this paper, we propose a conceptual framework to guide the design of a computer-supported collaborative learning intervention to help students learn how to improve collaborative knowledge building discourse at the level of the small group. The framework focuses on scripting individual and collective regulatory processes following collaboration.…

This paper describes the implementation and testing of our concept of adaptable rubrics, defined as analytical rubrics that arrange assessment criteria at multiple levels that can be expanded on demand. Because of its adaptable nature, these rubrics cannot be implemented in paper formats, neither are they supported by current Learning Management…

Public schools and districts in Nebraska are required to be accredited. The Nebraska Department of Education allows two methods to reach accreditation, the Nebraska Frameworks model and AdvancED Accreditation model. This study examined the factors that drive accreditation activities in Nebraska public schools and districts. It sought to answer…

Two main mechanisms, articulatory rehearsal and attentional refreshing, are argued to be involved in the maintenance of verbal information in working memory (WM). Whereas converging research has suggested that rehearsal promotes the phonological representations of memoranda in working memory, little is known about the representations that…

This paper presents the Latent Class Level-PCM as a method for identifying and interpreting latent classes of respondents according to empirically estimated performance levels. The model, which combines elements from latent class models and reparameterized partial credit models for polytomous data, can simultaneously (a) identify empirical…

Background/Context: From two decades of research on the grading practices of teachers in secondary schools, researchers discovered that teachers evaluated students on numerous factors that do not validly assess a student's achievement level in a specific content area. These consistent findings suggested that traditional grading practices evolved…

The purpose of this article is to introduce and review the capability and performance of the Stata item response theory (IRT) package that is available from Stata v.14, 2015. Using a simulated data set and a publicly available item response data set extracted from Programme of International Student Assessment, we review the IRT package from…

Observations and ratings of classroom teaching and interactions collected over time are susceptible to trends in both the quality of instruction and rater behavior. These trends have potential implications for inferences about teaching and for study design. We use scores on the Classroom Assessment Scoring System-Secondary (CLASS-S) protocol from…

Indigenous people of Dayak tribe in Kalimantan, Indonesia have traditionally relied on a system of mutual cooperation called "handep." The cultural context has an influence on students mathematics learning. The "handep" system might be suitable for modern learning situations to develop mathematical problem-solving skill. The…

Recent research on multiple measures of teaching effectiveness has redefined the role of in-classroom observations in teacher evaluation systems. In particular, most states now mandate that teachers are observed on multiple occasions during the school year, and it is increasingly common that multiple raters are utilized across the different rating…