This paper investigated consequences of measurement error in the pretest on the estimate of the treatment effect in a pretest-posttest design with the analysis of covariance (ANCOVA) model, focusing on both the direction and magnitude of its bias. Some prior studies have examined the magnitude of the bias due to measurement error and suggested…

The purpose of this study was to investigate whether simulated differential motivation between the stakes for operational tests and anchor items produces an invalid linking result if the Rasch model is used to link the operational tests. This was done for an external anchor design and a variation of a pretest design. The study also investigated…

Contribution: This paper presents the "PBL classroom model," an agent-based simulation (ABS) that allows testing of several scenarios of a project-based learning (PBL) application by considering different levels of soft-skills, and students' perception of the methodology. Background: While the community has made great advances in…

In this study, the effect of appended pretesting was evaluated with regard to item statistics and examinee scores for groups of items that were pretested as part of a large-scale operational testing program. In appended pretesting, items are administered in a separately timed section at the end of an operational test battery. Two evaluations were…

Research discussed in this paper was conducted as part of an ongoing large-scale simulation study to evaluate methods of calibrating pretest items for computerized adaptive testing (CAT) pools. The simulation was designed to mimic the operational CAT Armed Services Vocational Aptitude Battery (ASVAB) testing program, in which a single pretest item…

The accuracy with which regression models estimate treatment effects is dependent upon a number of conditions. The stability of the regression line (a function of sample size and correlation between pretest and posttest) is said to be the most important of these conditions. The utility of regression models is proportional to the size of the…

Recent research results strongly suggest that the theoretical problems of change measures have limited practical significance for measuring individual growth, and it is important to determine whether this is also the case for measuring school impact. Accordingly, in this study artifical data were used to assess the correlation between several…

A computer simulation procedure was developed to reproduce the overall pattern of results obtained in the Educational Testing Service Growth Study. Then simulated data for seven sets of 10,000 to 15,000 cases were analyzed, and findings compared on the basis of correlations between estimated and true growth scores. Findings showed that growth was…

The four main approaches to measuring treatment effects in schools; raw gain, residual gain, covariance, and true scores; were compared. A simulation study showed true score analysis produced a large number of Type-I errors. When corrected for this error, this method showed the least power of the four. This outcome was clearly the result of the…