Exploratory research with large numbers of variables and even larger numbers of relationships is likely to result in frequent Type II errors; that is falsely accepting an incorrect null hypotheses. There are three ways of reducing Type II errors: (1) choosing a lower significance level, which then increases the probability of Type I error; (2)…

The twin issues of reliability (consistency) and validity (relevance) of student ratings of teacher performance, and the overall issues of generalizability, continue to arise in assessing the usefulness of this faculty evaluation technique. Recent literature addressed four basic questions regarding these issues as they pertain to two faculty…

The author conducted a cross national analysis of sociological research reported in leading journals at two points in time over a ten year period to determine if sociologists' ability to produce valid social generalizations had improved significantly over the recent past. The official journals of the United States (American Sociological Review),…

In order to determine whether preschool children can successfully identify externalizing symptomatic behaviors (aggression and hyperactivity) in their male classmates, and whether children's perceptions are associated with peer-rated popularity and rejection, 154 preschool boys and girls were interviewed through the use of a peer-nominating…

The Joint Dissemination Review Panel (JDRP) examines educational interventions and determines their effectiveness. This study's purpose was to assess this review process and identify common characteristics of submittals and ways in which approvals and rejections differed. A locally-developed instrument which collected data on the intervention's…

A computer managed instruction (CMI) package to teach BASIC programming, together with questions to be answered in its evaluation, are described. The training package consists of 16 learning modules coupled; an interactive CMI system generates pretests and posttests for each module and branches students to the next appropriate module. It is…

Evaluation is viewed as essential to decision making and social policy development. Since conventional methods have been disappointing or inadequate, naturalistic inquiry (N/I) differs from conventional science in minimizing constraints on antecedent conditions (controls) and on output (dependent variables). N/I is phenomenological rather than…

Arguments have recently been put forth that standard textbook procedures for determining the sample size necessary to achieve a certain level of power in a completely randomized design are incorrect when the dependent variable is fallible because they ignore measurement error. In fact, however, there are several correct procedures, one of which is…

The possibility that "coder drift" may be a source of bias in classroom observation systems was investigated in two parallel experiments using the Flanders System of Interaction Analysis. Coder drift may be described as the differences in coding which result when observers create their own idiosyncratic systems of classifying teacher…

The results of two experiments support the hypothesis of "coder drift" which is defined as change that takes place while trained coders are using a system for a number of classroom observation sessions. The coding system used was a modification of the low-inference Flanders System of Interaction Analysis which calls for assigning…

THE PURPOSE OF THIS RESEARCH WAS TO STUDY THE EFFECT OF ERROR OF MEASUREMENT UPON THE POWER OF STATISTICAL TESTS. ATTENTION WAS FOCUSED ON THE F-TEST OF THE SINGLE FACTOR ANALYSIS OF VARIANCE. FORMULAS WERE DERIVED TO SHOW THE RELATIONSHIP BETWEEN THE NONCENTRALITY PARAMETERS FOR ANALYSES USING TRUE SCORES AND THOSE USING OBSERVED SCORES. THE…

Although the design of the writing task itself may present assessment problems, the scoring of the piece of writing raises the greatest difficulties for large-scale testing of writing ability. A study investigated whether teachers and staff members of the Southwest Regional Laboratory (SWRL) (1) scored the same way, (2) agreed with each other in…

A review of the research literature considers the following questions: (1) what is the general quality of the information resulting from systematic student evaluation of faculty effectiveness?; and (2) what are the consequences of soliciting this information and making it available to those people who may utilize it in their decision making…

A study was conducted at Los Angeles Pierce College (LAPC) to test the relative reliability of small-sample telephone interviewing and large-sample mailings with postcard and telephone reminders for student follow-up. The study involved conducting a telephone survey of 245 former students who had enrolled at LAPC in four program areas, and a mail…