This article develops a set-theoretic approach to Bayes's theorem and Bayesian process tracing. In the approach, hypothesis testing is the procedure whereby one updates beliefs by narrowing the range of states of the world that are regarded as possible, thus diminishing the domain in which the actual world can reside. By explicitly connecting…

To begin statistical analysis, Bayesians quantify their confidence in modeling hypotheses with priors. A prior describes the probability of a certain modeling hypothesis apart from the data. Bayesians should be able to defend their choice of prior to a skeptical audience. Collaboration between evaluators and stakeholders could make their choices…

This article argues that strong interrelations between methodological and theoretical advances exist. Progress in, especially comparative, methods may have important impacts on theory evaluation. By using the example of the "Varieties of Capitalism" approach and an international comparison of higher education systems, it can be shown…

Contradictory findings from "well-implemented" rigorous evaluations invite researchers to identify the differences that might explain the contradictions, helping to generate testable hypotheses for new research. This panel will examine efforts to ensure that the large number of local evaluations being conducted as part of four…

The purpose of this document is to describe the methods RAND used to analyze achievement for 23 personalized learning (PL) schools for the 2012-13 through 2013-14 academic years. This work was performed at the request of the Bill & Melinda Gates Foundation (BMGF), as part of a multi-year evaluation contract. The 23 schools were selected from a…

Background/Context: Principles of evidential reasoning have often been discussed in the context of educational and psychological measurement with respect to construct validity and validity arguments. More recently, Mislevy proposed the metaphor of assessment as an evidentiary argument about students' learning and abilities given their…

The motivation for this paper is the authors' recent work on several randomized control trials in which they found the primary result, which averaged across subgroups or sites, to be moderated by demographic or site characteristics. They are led to examine a distinction that the Institute of Education Sciences (IES) makes between "confirmatory"…

Research methods teaching in psychology is pivotal in preparing students for the transition from student as learner to independent practitioner. We took an action research approach to re-design, implement and evaluate a module guiding students through a programmatic and pragmatic research cycle. These revisions allow students to experience how…

This article discusses comparisons of means using exploratory and confirmatory approaches. Three methods are discussed: hypothesis testing, model selection based on information criteria, and Bayesian model selection. Throughout the article, an example is used to illustrate and evaluate the two approaches and the three methods. We demonstrate that…

Null hypothesis testing (NHT) is the most commonly used tool in empirical psychological research even though it has several known limitations. It is argued that since the hypotheses evaluated with NHT do not reflect the research-question or theory of the researchers, conclusions from NHT must be formulated with great modesty, that is, they cannot…

A search of the literature from the past 30 years reveals that there is a dearth of research surrounding effective interventions for intellectually gifted children in the early childhood years. The findings of 11 empirical studies of educational provisions for young gifted children were located and the methodological rigor of the studies examined.…

Developing students who can apply their knowledge of empirical research is a key outcome of the undergraduate psychology major. This learning outcome was assessed in two research methods courses by having students read and analyze a condensed empirical journal article. At the start and end of the semester, students in multiple sections of an…

For almost two decades, there has been growing interest in what design-based research (DBR) can contribute to both educational practice and theory. Since its introduction into the literature, this orientation to educational research has repeatedly been likened to aeronautical engineering as a way to clarify its nature and argue its potential. This…

In his article, "An Alternative to Null-Hypothesis Significance Tests," Killeen (2005) urged the discipline to abandon the practice of "p[subscript obs]"-based null hypothesis testing and to quantify the signal-to-noise characteristics of experimental outcomes with replication probabilities. He described the coefficient that he…

Lecoutre, Lecoutre, and Poitevineau (2010) have provided sophisticated grounding for "p[subscript rep]." Computing it precisely appears, fortunately, no more difficult than doing so approximately. Their analysis will help move predictive inference into the mainstream. Iverson, Wagenmakers, and Lee (2010) have also validated…