The power of a statistical test is, in part, a function of the reliability of the dependable variable being analyzed. The substitution of sigma square divided by the reliability coefficient for sigma is proposed. This enables the researcher to incorporate dependent variable reliability information when determining the sample size required for a…

Because of the duration and magnitude of this overall project, maintaining inter-rater reliability presented several problems. Since the study was designed to compare teacher behavior across a three-year time span, it was necessary to maintain reliability across that time period. The use of student raters posed additional problems of mobility and…

Two observers who were using an electronic digital data acquisition system were spot checked for reliability at random times over a four month period. Between-and within-observer reliability was assessed for frequency, duration, and duration-per-event measures of four infant behaviors. The results confirmed the problem of observer drift--the…

The differential Value Profile (DVP) was first published in the spring of 1963. The instrument grew out of the author's doctoral dissertation in which he reports its method of construction, some underlying psychological theory, its basic validation attempts, and internal consistency. Since its inception, the DVP has been used in many educational…

Scott's coefficient is an improved method for estimating observer reliability using any system which assigns events to mutually exclusive categories. The formula is the ratio of the actual difference between obtained and chance agreement to the maximum difference between obtained and chance agreement. A FORTRAN program for calculating this…

This research was designed to obtain information on the validity and reliability of three new evaluation techniques: (1) The Simulated Patient Management Problem (PMP), a written simulation exercise; (2) The Simulated Diagnostic Interview (DI), an oral exercise; and (3) The Simulated Proposed Treatment Interview (PTI), another oral exercise. These…

This study examined variation in transcriber disagreement as a function of transcriber's linguistic background, the transcription task, and the nature of judgment involved. Three linguistics students trained in phonetic transcription listened to the same tapes of Midwestern kindergarteners pronouncing lists of common words. Transcription task…

A study was conducted to observe the varying degree of consistency between three sources of data with respect to the same classroom phenomenon. Data was gathered in a public high school near Detroit in 1969. Using the basic 10-category Flanders interaction analysis system, systematic observations of controversial issues discussions were made in…

Basic to many psychological investigations is the question of agreement between observers who independently categorize people. Several recent studies have proposed measures of agreement when a set of nominal scale categories have been pre-defined and imposed on both observers. This study, in contrast, developes a measure of agreement for settings…

For instructional materials to be certified as "effective," students must meet instructional objectives operationalized by criterion tests. By implication, evaluators must agree when criteria are or are not met. Fourteen instructors evaluated 10 posterior bridges. Interjudge agreements for total bridges and individual attributes were low, as they…

An easy-to-use plan that offers the accuracy and objectivity of statistics without the complexities is described. The plan is an adaptation of a general statistical procedure called the Median Test. The 10-step computation of the Median Test is given. An appendix discusses the "median." (DB)

The Navy Vocational Interest Inventory (NVII) was developed to help recruits determine the ratings corresponding most closely with their vocational preferences and in which they would probably be most satisfied. Recently, 15 occupational scales were constructed for the NVII to reflect the degree of relationship between an individual's interests…

The purpose of this document was to present the development and validation of an instrument to be utilized by local inservice education coordinators in the evaluation of career education inservice training programs. It is proposed that the instrument, the Career Education Concept Inventory (CECI), can be used to provide a measure of understandings…

This study investigated two approaches for obtaining job ratings in order to determine which procedure provided the most stable ratings, given a constant number of raters. Specifically, it compared reliability estimates determined by averaging across individually obtained job ratings and reliability estimates based on consensus ratings from…

Described is the Quick Neurological Screening Test (QNST), a measure of "soft" neurological signs frequently associated with learning disabilities, and discussed are research findings on the QNST. The QNST is said to provide classroom teachers with a method for identifying children whose learning disabilities are caused by neurological…