Construct validation theory is founded upon conflicting metaphysical principles, methodological approaches, and standards of adequacy. This paper explores these unrecognized conflicts and indicates some consequences that these conflicts have for construct validation theory. (Author/PN)

Possible bias due to sampling problems or low response rates has been a troubling "nuisance" variable in empirical research since seminal and classical studies were done on these problems at the beginning of this century. Recent research suggests that: (1) earlier views of the alleged bias problem were misleading; (2) under a variety of fairly…

In discussing the problems of evaluating Canadian bilingual immersion programs, six related aspects are separately considered: (1) the research design, (2) the students tested, (3) the nature of the tests used, (4) the methods of analysis employed, and (5) the interpretation of findings. (Author/SJL)

The biasing effects of a pretest on subsequent post-test results were investigated in two experimental studies. In general, the results argue for using designs without pretests. (Author/JKS)

Twenty-six papers concerning Elementary and Secondary Education Act Title I program evaluation--presented at the 1978 American Educational Research Association/National Council on Measurement in Education meeting--are summarized. Speakers from the federal government are: C.T. Cross (see ED 156 695); R.L. Fishbein; and D. Ozenne. State department…

Discusses the need for awareness of real and potential problems in educational evaluation approaches, kinds of problems there are with true experimental designs and with quasi-experimental designs, and the criteria applied by the Joint Dissemination Review Panel to its review of educational programs. (MF)

Differences in 164 fifth grade students' reading comprehension scores were obtained with four different tasks typically used to measure comprehension (multiple choice, recall, cloze, and maze), using four different reading passages that were equated according to readability formulas. Data analyses revealed significant effects for passage, task,…

Threats to the validity of the Title I Evaluation and Reporting System are covered in two parts: an annotated bibliography of reports concerned with technical issues, and a discussion of threats to validity--from the reporting system in general and from each model in particular. Threats common to all three evaluation models are reported:…

In 1987, a survey report by J. Cannell was published criticizing state reports based on national standardized testing of elementary school students that indicated that all 50 states were above the national average. This phenomenon--known as the "Lake Wobegon Effect"--is analyzed. Technical problems with the study are considered. (TJH)

Twelve common errors which can invalidate an otherwise sound evaluation are identified, and ways to avoid them are presented. The hazards are: (1) grade-equivalent scores; (2) inappropriate statistical adjustments with nonequivalent control groups; (3) administering norm-referenced tests at inappropriate times of the school year; (4) inappropriate…

Many people hope that the unique skills and perspectives of anthropologists can help them to make the contributions to social programs that have eluded other social scientists. Ethnography promises to make a significant contribution to evaluation of educational innovations. Before this promise can be realized, however, there are many practical and…

One of the overriding problems of any psychometric study is that aspects of behavior that are nonquantifiable or difficult to quantify are not considered. The other side of this coin is that those aspects of behavior that are easiest to quantify tend to be what are examined. In evaluating bilingual education programs, problems are found in the…

The increasing use of educational performance or outcome measurements for a range of policy purposes points to new procedures for adjusting data for population composition. The purposes include: program formulation, budget resource allocation, grant-in-aid designs, performance incentive payments, consumer information for school selection, and…

This review of the literature addresses scientific and methodological concerns of research with children from various racial and ethnic backgrounds in the United States. It first identifies three major demographic trends: (1) the increasingly multiethnic and multilingual nature of American society; (2) the increasing number of children in poverty;…

At the 1989 meeting of the National Faculty of the Center for Research on Evaluation, Standards, and Student Testing (CRESST), faculty members were invited to present testimony on what they viewed as the most pressing research and policy issues in the fields of testing, evaluation, and standards. These views are expressed in this document,…