We revisit the merits of the retrospective pretest-posttest (RPP) design for repeated-measures research. The underutilized RPP method asks respondents to rate survey items twice during the same posttest measurement occasion from two specific frames of reference: "now" and "then." Individuals first report their current attitudes…

We explore the conditions under which short, comparative interrupted time-series (CITS) designs represent valid alternatives to randomized experiments in educational evaluations. To do so, we conduct three within-study comparisons, each of which uses a unique data set to test the validity of the CITS design by comparing its causal estimates to…

Finding ways to improve student success in calculus is a critically important step on the path to supporting students who are pursuing degrees in STEM fields. Far too many students fail calculus 1 and are pushed to drop their majors in technical fields. One way of addressing this issue is by following a program that was pioneered at University of…

The purpose of this study was to develop a scoring rubric to assess students' manipulation skills and identify students' learning difficulties in conducting organic chemistry experiments. In constructing the scoring rubric, we first analyzed the skills needed in the experiment, then divided the skills into subskills, and finally…

The current study was designed to extend the current literature to study scale drift in CAT as part of improving quality control and calibration process for ACCUPLACER, a battery of large-scale adaptive placement tests. The study aims to evaluate item parameter drift using empirical data that span four years from the ACCUPLACER Arithmetic…

Automated scoring models for the "e-rater"® scoring engine were built and evaluated for the "GRE"® argument and issue-writing tasks. Prompt-specific, generic, and generic with prompt-specific intercept scoring models were built and evaluation statistics such as weighted kappas, Pearson correlations, standardized difference in…

This paper addresses the use of computer-based testing in distance education, based on the experience of Universitas Terbuka (UT), Indonesia. Computer-based testing has been developed at UT for reasons of meeting the specific needs of distance students as the following: (1) students' inability to sit for the scheduled test; (2) conflicting test…

Describes how Pre-Planning Evaluation (PPE) aids initial curriculum planning. Presents a case study of how PPE influenced the development of the electricity curriculum in Israeli technical high schools. Concludes that PPE is an essential component of the curriculum development process. (GEA)

The development of a project to assess the effectiveness of schools in a large urban school district is described in this paper, with a focus on improving the validity of school effectiveness instruments. Purposes of the project were to develop a methodology for reporting effective school strategies and to generate a broader profile of schools'…

The biasing effects of a pretest on subsequent post-test results were investigated in two experimental studies. In general, the results argue for using designs without pretests. (Author/JKS)

The problem of the appropriate comparison group for bereavement studies is discussed. The use of the retrospective pretest design is illustrated through a longitudinal study of bereavement outcome following pregnancy loss, involving 138 bereaved women and a comparison group of 215 currently pregnant women. (SLD)

As a step in combining three of the Elementary and Secondary Education surveys of the National Center for Education Statistics to produce one database linking local education agencies, schools, and teachers, the merged procedures and collections forms were pretested. Different methods were tested by splitting the pretest sample of 220 public…

Although actual or projected satisfaction with parenting is important in determining whether a couple will become parents and how large their family will be, only minimal research has assessed parental satisfaction. The Cleminshaw-Guidubaldi Parent Satisfaction Scale, a 50-item Likert-type instrument designed to measure components of satisfaction…

The accuracy with which regression models estimate treatment effects is dependent upon a number of conditions. The stability of the regression line (a function of sample size and correlation between pretest and posttest) is said to be the most important of these conditions. The utility of regression models is proportional to the size of the…

Two types of sensitivity indices were developed in this paper, one internal to the total test and the second external. To evaluate the success of these statistics the three criteria suggested for a satisfactory index of item quality were considered. The Internal Sensitivity Index appears to meet these demands. Certainly it is easily computed. In…