Recent methods to improve generalizations from nonrandom samples typically invoke assumptions such as the strong ignorability of sample selection, which is challenging to meet in practice. Although researchers acknowledge the difficulty in meeting this assumption, point estimates are still provided and used without considering alternative…

Data have long been considered a key factor in organizational decision-making (Simon, 1955; Lindblom & Cohen, 1979). Data offer perspective, guidance, and insights that inform policy and practice (Newell & Simon, 1972; Kennedy, 1984). Recently, education policymakers have invested in the use of data for organizational improvement in states…

In large-scale assessments, such as state-wide testing programs, national sample-based assessments, and international comparative studies, there are many steps involved in the measurement and reporting of student achievement. There are always sources of inaccuracies in each of the steps. It is of interest to identify the source and magnitude of…

The utility of value-added estimates of teachers' effects on student test scores depends on whether they can distinguish between high- and low-productivity teachers and predict future teacher performance. This article studies the year-to-year variability in value-added measures for elementary and middle school mathematics teachers from five large…

This paper helps researchers develop criteria for conducting surveys. Cost considerations frame the paper's suggestions. Specifically, the text advises that researchers consider the human and material resources needed, type of survey instrument, medium for conducting the survey (mail, telephone, personal interview, etc.), data analysis…

Suggestions for adapting NAEP procedures for local use are based on the assumptions that instructional decisions can be improved through systematic gathering of data and that every student is not tested. (Author/KM) Aspect of National Assessment (NAEP) dealt with in this document: Procedures (Utilization).

When evaluating the effectiveness of an educational program, multiple matrix sampling is particularly effective and efficient when the goal of the evaluation is estimating group (as opposed to individual) performance. The technique is described in some detail, with its advantages and disadvantages, and examples of its application are given.…

Describes the evolution of national educational assessment in the United States, its present stage of development, methodological and reporting problems, and approaches applicable to assessment data. Examines how proposed approaches can be applied in the California Assessment Program and in the National Assessment of Educational Progress. (MJL)

An alternative method for analyzing and evaluating prototype and revised instructional material is compared to traditional methods. The procedure and three approaches of sequential analysis are presented and advantages are discussed. References are included. (BK)

Procedures for implementing the Pennsylvania Plan (see TM 000 608) are detailed, and the method of achieving representative sampling is described. Functions of school representatives involved in the data collection are listed, and their training is outlined. Procedures for collecting school and teacher information, and administering tests and…

A number of quality assurance educational evaluation methods are illustrated. Evaluation data obtained from the Providence, Rhode Island, school district are used. The methods are: (1) from auditing, internal control; (2) from accounting, double bookkeeping; and (3) from industrial quality control, acceptance sampling and cumulative percentage…

This article considers various kinds of probability and non-probability samples in both experimental and survey studies. Throughout, how a sample is chosen is stressed. Size alone is not the determining consideration in sample selection. Good samples do not occur by accident; they are the result of a careful design. (Author/JAZ)

One of the benefits of performance based assessment is its allowance for the teacher to gather information about how the student understands and applies knowledge. In this Digest, the basic steps a teacher should take to plan and execute effective performance-based assessment are outlined. Coverage includes the steps of planning and executing…

Georgia's Needs Assessment Package, which has many sample forms, includes a procedure which helps to identify short-range and long-range educational needs. This procedure utilizes community participation which broadens the decision-making base and serves to legitimatize actual decisions. It utilizes the opinions of professional educators,…

The effects of certain characteristics (e.g., sample design) of National Assessment of Educational Progress (NAEP) data on statistical analysis techniques are considered. Ignoring special features of NAEP data and proceeding with a standard analysis can produce inferences that underestimate the true variability and overestimate the true degrees of…