In this article, we pursue two points of discussion. First, a new illustration is presented of the person-oriented tenet according to which it can be hazardous to generalize to the individual results that are based on the analysis of aggregated data. Second, it is illustrated that taking into account serial dependence information can result in not…

Research in psychology and education tend to use large-N group designs that necessitate reporting of mean measures analyzed mostly with null hypothesis statistical testing (NHST), but sometimes with Bayesian, or the estimation approaches in inferential statistics. These approaches all render the average person or student as the the putative…

Conceptual replications have received increased coverage in the educational research agenda. This article argues for clarity in, and justification of, the definition, scope, and boundaries of a conceptual replication and what it can and cannot do. It argues for clear justifications when changing components from those of the original study. The…

Experimental studies of educational interventions are rarely based on representative samples of the target population. This simulation study tests two formal sampling strategies for selecting districts and schools from within strata when they may not agree to participate if selected: (1) balanced selection of the most typical district or school…

This paper aims to unsettle assumptions of generalizability and representativeness in quantitative research using queer framings and positionalities. We argue that generalizability and representativeness are tools of supremacist dominance that reinforce harmful and essentialist categories of identities for the false purpose of statistical…

Observations are widely used in research and evaluation to characterize teaching and learning activities. Because conducting observations is typically resource intensive, it is important that inferences from observation data are made confidently. While attention focuses on interrater reliability, the reliability of a single-class measure over the…

Students often believe that statistical significance is the only determinant of whether a quantitative result is "important." In this paper, I review traditional null hypothesis statistical testing to identify what questions inferential statistics can and cannot answer, including statistical significance, effect size and direction,…

The authors present two sets of replication studies and aim to confirm, refute, and expand prior work.

This resource has been written specifically for higher education practitioners who are interested in improving students' learning experiences through the process of researching their own practice. We use the term 'higher education practitioners' to describe all who work in universities and who have a stake in students' learning experiences.…

Special education researchers conduct studies that can be considered replications. However, they do not often refer to them as replication studies. The purpose of this article is to consider the potential benefits of conceptualizing special education intervention research within a framework of systematic, conceptual replication. Specifically, we…

Many studies in mathematics education research occur with a nonrepresentative sample and are never replicated. To challenge this paradigm, I designed a large-scale study evaluating student conceptions in group theory that surveyed a national, representative sample of students. By replicating questions previously used to build theory around student…

The present paper aims to discuss how data from international large-scale assessments (ILSAs) can be utilized and combined, even with other existing data sources, in order to monitor educational outcomes and study the effectiveness of educational systems. We consider different purposes of linking data, namely, extending outcomes measures,…

In their examination of the effectiveness of affect regulation strategies, Webb, Miles, and Sheeran (2012) offered the results of a broad meta-analysis of studies on regulatory interventions. Their analysis provides an alternative to our earlier, more focused meta-analysis of the affect regulation literature (Augustine & Hemenover, 2009).…

The article discusses comparative organizational studies, focusing on change in higher education institutions, and in particular (but not only) on studies that compare organizations in different national settings. It first presents and discusses different approaches to comparison based on a couple of typologies that have been developed to identify…

There appears to be a mounting consciousness in academia that knowledge production and the scholarly dissemination of knowledge do not necessarily lead to general well-being or improvement in society. In this article we start with ourselves by initiating an exploration into generative possibilities for becoming agents of social change through our…