In light of the replication crisis in psychology, null-hypothesis significance testing (NHST) and "p"-values have been heavily criticized and various alternatives have been proposed, ranging from slight modifications of the current paradigm to banning "p"-values from journals. Since the physics education research community…

Research studies in Educational Data Mining (EDM) often involve several variables related to student learning activities. As such, it may be necessary to run multiple statistical tests simultaneously, thereby leading to the problem of multiple comparisons. The Benjamini-Hochberg (BH) procedure is commonly used in EDM research to address this…

Evidence-based assessment (EBA) requires that investigators employ scientific theories and research findings to guide decisions about what domains to measure, how and when to measure them, and how to make decisions and interpret results. To implement EBA, investigators need high-quality assessment tools along with evidence-based processes. We…

Background: Scientific thinking is an essential learning goal of science education and it can be fostered by inquiry learning. One important prerequisite for scientific thinking is procedural understanding. Procedural understanding is the knowledge about specific steps in scientific inquiry (e.g. formulating hypotheses, measuring dependent and…

BASIE is a framework for interpreting impact estimates from evaluations. It is an alternative to null hypothesis significance testing. This guide walks researchers through the key steps of applying BASIE, including selecting prior evidence, reporting impact estimates, interpreting impact estimates, and conducting sensitivity analyses. The guide…

In 1881, Donald MacAlister posed a problem in the "Educational Times" that remains relevant today. The problem centers on the statistical evidence for the effectiveness of a treatment based on a comparison between two proportions. A brief historical sketch is followed by a discussion of two default Bayesian solutions, one based on a…

Multilevel models are often used to evaluate hypotheses about relations among constructs when data are nested within clusters (Raudenbush & Bryk, 2002), although alternative approaches are available when analyzing nested data (Binder & Roberts, 2003; Sterba, 2009). The overarching goal of this article is to suggest when it is appropriate…

Online education is rapidly becoming a significant method of course delivery in higher education. Consequently, instructors analyze student performance in an attempt to better scaffold student learning. Learning analytics can provide insight into online students' course behaviors. Archival data from 167 graduate level education students enrolled…

The crisis caused by the COVID-19 virus has far-reaching effects in the field of education, as schools were closed in March 2020 in many countries around the world. In this article, we present and discuss the School Barometer, a fast survey (in terms of reaction time, time to answer and dissemination time) that was conducted in Germany, Austria…

As scientific models of student thinking, learning progressions (LPs) have been evaluated in terms of one important, but limited, criterion: fit to empirical data. We argue that LPs are not empirically adequate, largely because they rely on problematic assumptions of theory-like coherence in students' thinking. Through an empirical investigation…

This paper outlines a novel approach to engineering education research that provides three dimensions of learning through an experiential class activity. A simulated decision activity brought current research into the classroom, explored the effect of experiential activity on learning outcomes and contributed to the research on innovation decision…

Learning analytics and educational data mining are introducing a number of new techniques and frameworks for studying learning. The scalability and complexity of these novel techniques has afforded new ways for enacting education research and has helped scholars gain new insights into human cognition and learning. Nonetheless, there remain some…

The acceptance of Educational Data Mining (EDM) technology is on the rise due to, its ability to extract new knowledge from large amounts of students' data. This knowledge is important for educational stakeholders, such as policy makers, educators, and students themselves to enhance efficiency and achievements. However, previous studies on EDM…

When validating assessment models built with data mining, generalization is typically tested at the student-level, where models are tested on new students. This approach, though, may fail to find cases where model performance suffers if other aspects of those cases relevant to prediction are not well represented. We explore this here by testing if…

This article discusses the application of logit models for the analyses of 2-way categorical observations. The models described are generalized linear models using the logit link function. One of the models is the Rasch model (Rasch, 1960). The objective is to test hypotheses of marginal and conditional independence between explanatory quantities…