Traditionally, cluster randomized controlled trials are analyzed with the average intervention effect of interest. However, in populations that contain higher degrees of heterogeneity or variation may differ across different values of a covariate, which may not be optimal. Within education and social science contexts, exploring the variation in…

In this study we examined the relationship of students' 21st century skills with their academic and behavioral outcomes. We investigated K-12 teachers' (n = 150) judgment of students' (n = 3,108) use of 21st century skills (i.e., persistence, curiosity, affect, and cognition behaviors) via a Likert-type rating scale. We compared teachers' ratings…

The goal of this paper is to examine the relationship of student motivation and achievement in low-stakes assessment contexts. Using Pearson product-moment correlations and hierarchical linear regression modelling to analyse data on 794 tertiary students who undertook a low-stakes engineering mechanics assessment (along with the questionnaire of…

This study investigated the relationship of kindergarten teachers' ratings of their students' 21st century skills (college readiness skills) with students' behavioral and academic performance. Teachers rated the frequency that their students (n = 579) demonstrated persistence, curiosity, affective, and cognitive (e.g., critical thinking) behaviors…

Given the wide use of peer assessment, especially in higher education, the relative accuracy of peer ratings compared to teacher ratings is a major concern for both educators and researchers. This concern has grown with the increase of peer assessment in digital platforms. In this meta-analysis, using a variance-known hierarchical linear modelling…

Item response theory (IRT) models are the main psychometric approach for the development, evaluation, and refinement of multi-item instruments and scaling of latent traits, whereas multilevel models are the primary statistical method when considering the dependence between person responses when primary units (e.g., students) are nested within…

The factorial survey is an experimental design consisting of varying situations (vignettes) that have to be judged by respondents. For more complex research questions, it quickly becomes impossible for an individual respondent to judge all vignettes. To overcome this problem, random designs are recommended most of the time, whereas quota designs…

Introduction: Despite the fact that the capacity of non-written sources (NWS) understanding is included in the Spanish secondary education curriculum, no research has addressed this competence. The main objective of this study is analysing how teachers consider NWS understanding that should be assessed to help decide which kind of support give to…

In this study, it is aimed to examine the effect of classroom assessment on science and mathematics achievements. For this purpose, hierarchical linear modeling (HLM) is performed using variables of like learning science/maths, engage teaching in science/maths, confidence in science/maths, and home resources for learning variables at the student…

We present a multi-level model of contextual factors that may influence teachers' assessment practices, and use this model in a selected review of existing literature on teachers' assessment knowledge, views and conceptions with respect to these contextual factors. Adapting Kozma's model, we distinguish three levels of influence on teachers'…

We explored the feasibility of using automated scoring to assess upper-elementary students' reading ability through analysis of transcripts of students' small-group discussions about texts. Participants included 35 fourth-grade students across two classrooms that engaged in a literacy intervention called Quality Talk. During the course of one…

For 30 years, researchers have investigated oral reading fluency as a measure of growth in reading proficiency. Yet, little research has been done with these measures in the context of progress monitoring in Tier 2 systems. First, we document teachers' progress-monitoring decisions on type of passage (on-grade or off-grade) and how often to…

The Objective Structured Clinical Exam (OSCE) is a widely used method of assessment in medical education. Rater cognition has become an important area of inquiry in the medical education assessment literature generally, and in the OSCE literature specifically, because of concerns about potential compromises of validity. In this study, a novel…

In educational measurement, responses of students on items are used not only to measure the ability of students, but also to evaluate and compare the performance of schools. Analysis should ideally account for the multilevel structure of the data, and school-level processes not related to ability, such as working climate and administration…

Assessment for Learning (AfL), as a way to promote learning, requires a "match" or a shared focus between student and teacher to be effective. But students and teachers may differ in their perceptions of the purpose and process of classroom assessment meant to promote learning. Perceptions regarding AfL practices in their classroom were…