Research Reproducibility: Educating for Reproducibility, Pathways to Research Integrity was an interdisciplinary, conference hosted virtually by the University of Florida in December 2020. This event brought together educators, researchers, students, policy makers, and industry representatives from across the globe to explore best practices,…

Randomized trials play an important role in estimating the effect of a policy or social work program in a given population. While most trial designs benefit from strong internal validity, they often lack external validity, or generalizability, to the target population of interest. In other words, one can obtain an unbiased estimate of the study…

Single-case researchers frequently adopt a form of response-guided experimentation where decisions about the design of the study are made based on an ongoing visual analysis. For example, multiple-baseline researchers may delay the start of intervention until data document a stable baseline pattern so that baseline trends can be reliably extended,…

In this paper the authors share their insights on whether it is necessary to articulate a research methodology when reporting on theoretical research. Initially the authors, one being a supervisor and the other, a PhD student and a colleague, were confronted with the question during supervision and writing of a thesis on theoretical research.…

Randomized control trials are considered the gold standard for conducting research and estimating causal effects; however, educational research rarely lends itself to experimental design and true randomization. In recent years, there has been a growing interest in finding new approaches to estimate causal effects in nonrandomized studies in…

Methods for meta-analyzing single-case designs (SCDs) are needed in order to inform evidence based practice in special education and to draw broader and more defensible generalizations in areas where SCDs comprise a large part of the research base. The most widely used outcomes in single-case research are measures of behavior collected using…

My research is rooted in improving K-12 educational practice using motivational facets made possible through adaptive tutoring systems. In an attempt to isolate best practices within the science of learning, I conduct randomized controlled trials within ASSISTments, an online adaptive tutoring system that provides assistance and assessment to…

What began with a few libraries' early application of ethnographic methods to learn more about user behaviors and needs has grown to become a significant body of work done across many institutions using a broad range of methods. User-centered investigations are increasingly influential in discussions about the shape and future of the research…

Institutional performance benchmarking requires identifying a set of reference or comparator institutions. This paper describes a method by which an institution can identify other institutions that are most similar to itself using a methodology that identifies the nearest institutional neighbors based on a balanced set of metrics accessed from…

As lines between research paradigms continue to blur with the ever-increasing popularity of mixed methods research, there are useful, and occasionally oxymoronic, opportunities for educational researchers to juxtapose tools from opposing methods. The gold standard is just not possible in so much of what we do with small-scale research, nor is it…

Identifying an area of research a topic, deciding on a problem, and formulating it in to a researchable question are very difficult stages in the whole research process at least for beginners. Few books on research methodology elaborates the various process involved in problem selection and clarification. Viewing research and problem selection as…

Experimental designs are distinguished as the best method to respond to questions involving causality. The purpose of the present paper is to explicate the logic of experimental design and why it is so vital to questions that demand causal conclusions. In addition, types of internal and external validity threats are discussed. To emphasize the…

An experiment is deemed to be valid, inasmuch as valid cause-effect relationships are established, if the results are due only to the manipulated independent variable (possess internal validity) and are generalizable to groups, environments, and contexts outside of the experimental settings (possess external validity). Consequently, all…

The increased interest in reporting effect sizes means that it is necessary to consider what should be included in a primer on effect sizes. A review of papers on effect sizes and commonly repeated statistical analyses suggests that it is important to discuss effect sizes relative to bivariate correlation, t-tests, analysis of variance/covariance,…

The current pace of technological advance has provided an unprecedented opportunity to use innovative simulated tasks in computerized assessment. A primary challenge for the successful use of innovation in assessment rests with the application of sound principles of design to produce a valid assessment. An additional challenge is to maximize the…