Changing criterion designs (CCD) are single-case experimental designs that entail a step-by-step approximation of the final level desired for a target behavior. Following a recent review on the desirable methodological features of CCDs, the current text focuses on an analytical challenge: the definition of an objective rule for assessing the…

In the field of special education there is a dearth of group experimental studies that establish evidence-based practice. The effort to establish evidence-based practice has been associated with emphasizing experiments by using randomized controlled trial with large numbers of participants who are randomly assigned to a treatment. However,…

Experimentation is the foundation of science and an important process for students to understand and experience. However, it can be difficult to teach some aspects of experimentation within the time and resource constraints of an academic semester. Interactive models can be a useful tool in bridging this gap. This freely accessible simulation…

We propose a hierarchical Bayesian model for analyzing multi-site experimental fMRI studies. Our method takes the hierarchical structure of the data (subjects are nested within sites, and there are multiple observations per subject) into account and allows for modeling between-site variation. Using posterior predictive model checking and model…

Experiments allow researchers to randomly vary the key manipulation, the instruments of measurement, and the sequences of the measurements and manipulations across participants. To date, however, the advantages of randomized experiments to manipulate both the aspects of interest and the aspects that threaten internal validity have been primarily…

The slope of the z-transformed receiver-operating characteristic (zROC) in recognition memory experiments is usually less than 1, which has long been interpreted to mean that the variance of the target distribution is greater than the variance of the lure distribution. The greater variance of the target distribution could arise because the…

A course design for first-year science students is described, where the focus is on the skills necessary to do science. The course uses original research projects, designed by the students, to teach a variety of skills including reading the scientific literature, hypothesis development and testing, experimental design, data analysis and…

Background: Coaches developing young talent in team sports must maximise practice and learning of essential game skills and accurately and continuously assess the performance and potential of each player. Relative age effects highlight an erroneous process of initial and on-going player assessment, based largely on subjective opinions of game…

Background: Specificity and transfer of learning have been examined in experimental studies. However, their findings may not be relevant to practitioners because of the difference between the experiment conditions and teaching situations. This case study investigates the theoretical issue of specificity vs. transfer of learning by conducting…

A common mistake in analysis of cluster randomized experiments is to ignore the effect of clustering and analyze the data as if each treatment group were a simple random sample. This typically leads to an overstatement of the precision of results and anticonservative conclusions about precision and statistical significance of treatment effects.…

Mathematics education researchers conducting instruction experiments using a design research methodology are challenged with the analysis of often complex and large amounts of qualitative data. In this paper, we present two case studies that show how multimedia analysis software can greatly support video data analysis and theory development in…

This poem, with stanzas in limerick form, refers humorously to the many threats to validity posed by problems in research design, including problems of sample selection, data collection, and data analysis. (SLD)

Issues of reliability and validity in ethnographic design are compared to their counterparts in experimental design. Threats to the credibility of ethnographic research are summarized and categorized. Strategies intended to enhance credibility (study design, data collection, data analysis, and presentation of findings) are incorporated throughout…

Research supported by a grant from the National Science Foundation to the University of Texas at Austin. (DD)

Tests the validity of hypotheses concerning the effects of films on opinion formation. Reports on experiments using two third-grade classes and a conservation film produced for primary-school-age children. (Author/RK)