Until now, the conventional approach using two distinct groups, experimental and control, continues to dominate research, especially education research. Researchers, particularly those who are active in this domain, readily recognise this pattern when surveying literature. This article explores the use of the Solomon four-group design as a…

The current study examined whether the learning benefits of pretesting--like those produced by posttesting--generalize to classroom settings, and whether such benefits transfer to non-pretested related information. Before some lectures but not others, undergraduate students enrolled in a large research methods class were given a brief competitive…

We explore the conditions under which short, comparative interrupted time-series (CITS) designs represent valid alternatives to randomized experiments in educational evaluations. To do so, we conduct three within-study comparisons, each of which uses a unique data set to test the validity of the CITS design by comparing its causal estimates to…

This article summarizes results from 12 empirical evaluations of observational methods in education contexts. We look at the performance of three common covariate-types in observational studies where the outcome is a standardized reading or math test. They are: pretest measures, local geographic matching, and rich covariate sets with a strong…

The aim of this case study is to highlight the importance of piloting or pre-testing the research design (in this case, semi-structured interviews and narratives). Most textbooks offer minimal guidance about pre-testing the research design, and published reports rarely report whether the research was piloted and, if so, what were the results.…

In this study, we investigated whether incorporating eye tracking into cognitive interviewing is effective when pretesting survey questions. In the control condition, a cognitive interview was conducted using a standardized interview protocol that included pre-defined probing questions for about one-quarter of the questions in a 52-item…

There is a need for a holistic usability evaluation framework that accommodates social constructivist online courses. Social knowledge construction may not be adequately evaluated using current frameworks. This qualitative research study examined the usability needs of a social constructivist online course. Data from an online course were analyzed…