We consider estimating the effect of a treatment on a given outcome measured on subjects tested both before and after treatment assignment in observational studies. A vast literature compares the competing approaches of modelling the post-test score conditionally on the pre-test score versus modelling the difference, namely, the gain score. Our…

The purpose of this study was to compare calibration and linking methods for placing pretest item parameter estimates on the item pool scale in a 1-3 computerized multistage adaptive testing design in terms of item parameter recovery. Two models were used: embedded-section, in which pretest items were administered within a separate module, and…

An operational multistage adaptive test (MST) requires the development of a large item bank and the effort to continuously replenish the item bank due to concerns about test security and validity over the long term. New items should be pretested and linked to the item bank before being used operationally. The linking item volume fluctuations in…

The COVID-19 pandemic kick-started a new paradigm shift in the way and manner teaching and learning takes place on the global scale due to the lockdown that succeeded in shutting down school activities. As a result, most institutions turned to technology supported e-learning as against the traditional face-to-face teaching and learning in order to…

Experimental designs in the social sciences have received increasing attention due to their power to produce causal inferences. Nevertheless, experimental research faces limitations, including limited external validity and unrealistic treatments. We propose combining qualitative fieldwork and experimental design iteratively--moving back-and-forth…

The field of geometrical optics features a broad knowledge base concerning conceptual learning difficulties and student conceptions. However, in contrast to other areas of physics, no widely used standard test instrument exists to assess middle school students' conceptual knowledge of geometrical optics. This is even more true for other areas of…

The current study examined whether the learning benefits of pretesting--like those produced by posttesting--generalize to classroom settings, and whether such benefits transfer to non-pretested related information. Before some lectures but not others, undergraduate students enrolled in a large research methods class were given a brief competitive…

The researcher, who pretests a questionnaire conventionally, is confident that a small number of interviews reveals numerous problems afflict the questionnaire. The standardized interviewer is asked to observe respondent's verbal behaviors in order to realize whether respondent misunderstands question. But this strategy fails when respondent does…

This paper investigated consequences of measurement error in the pretest on the estimate of the treatment effect in a pretest-posttest design with the analysis of covariance (ANCOVA) model, focusing on both the direction and magnitude of its bias. Some prior studies have examined the magnitude of the bias due to measurement error and suggested…

Recent studies suggest that testing on prior material enhances subsequent learning of new material. Although such forward testing effect has received extensive empirical support, it is not yet clear how testing facilitates subsequent learning. One possible explanation suggests that interim testing informs learners about the format of an upcoming…

In spring 2016, a course at Ohio State University (OSU) was introduced to teach advanced agronomic concepts and included the option to prepare students for the Certified Crop Adviser (CCA) exams offered by the American Society of Agronomy. From 2019 to 2021, revisions in instructional methods were implemented that introduced the use of pretests…

Gateway science courses are an ongoing obstacle to recruitment into STEM fields (Science, Technology, Engineering, and Mathematics). Students come into courses that have a high cognitive load, which makes success in the course challenging. Instructional design can be used to reduce cognitive load. The findings demonstrate that leveraging pre- and…

One approach to encourage productive study strategies is to incorporate preparatory quizzes (or pre-quizzes) in which students are required to submit answers to questions before the underlying material is covered in class. In the present study, students took an introductory mechanical engineering class that either included pre-quizzes (treatment…

Considerable research has examined the prevalence and apparent consequences of task-unrelated thoughts (TUTs) in both laboratory and authentic educational settings. Few studies, however, have explored methods to reduce TUTs during learning; those few studies tested small samples or used unvalidated TUT assessments. The present experimental study…

Analyzing heterogeneous treatment effects (HTEs) plays a crucial role in understanding the impacts of educational interventions. A standard practice for HTE analysis is to examine interactions between treatment status and preintervention participant characteristics, such as pretest scores, to identify how different groups respond to treatment.…