The reliability and validity of teacher ratings, the relationship between teacher ratings and principal self-reports of instructional leadership, and the degree to which they are influenced by demographic factors are examined in this study. Methodology involved completion of the Instructional Leadership Inventory, a self-report measure, by 81…

This research aimed to investigate the effect of computers on the test and inter-rater reliability of writing test scores of ESL learners. Writing samples of 20 pen-paper and 20 computer group students were scored in analytic scoring method by two scorers, and then the scores were analyzed in Alpha (Cronbach) model. The results showed that the…

A total of 2,000 essays written by 1,000 students was submitted to generalizability analyses for domain-referenced tests. Each student had written one essay on each of two prompts representing two models of discourse. Each essay was read by six readers and judged on a scale of from 1 to 4. No reader read essays from both prompts. Reader agreement…

This report shows how Beta weights for the J-Coefficient may be easily developed without a formal validity study, and indicates how indications of ability other than tests can be used to measure the same abilities that are measured by tests. See also TM 001 163-64,166 for further information on job elements (J-Scale) procedures. (Author/DLG)

Perhaps more than ever before, college teaching is being studied and evaluated. This paper describes the development of a simple descriptive instrument used to focus observers' classifications and ratings of college teachers' instructional behaviors as recorded on video tape. The need for such an instrument is reviewed, the methodology for testing…

This study examined whether variations in the Developmental Observation Checklist (DC) format influences congruence of scores among both parents and the child's teacher. The DC was varied by adding pictorial illustrations and examples and having three response categories instead of two. Results from 100 sets of participants were evaluated with…

The Child Behavior Checklist and related forms were completed for 171 boys referred for school-based assessment resulting from academic and/or behavioral problems. Adolescents consistently underreported behavioral problems relative to parents and teachers regardless of subsequent diagnosis. Implications of these discrepancies in school-based…

To assess cross-observer agreement on personality profiles, an Index of Profile Agreement and an associated coefficient are proposed that take into account both the difference between the ratings and the extremes of their mean. Data from the Revised NEO Personality Inventory for 250 peer ratings/self-reports and 68 spouse ratings/self-reports…

Investigated the reliability and item appropriateness, as discerned by adults affiliated with an infant center, of the Caregiver-Environment Scale (CES). Found the CES to be an easy to use, reliable instrument for evaluation. (Author/SD)

This study explored the use of the Formal Elements Art Therapy Scale (FEATS) with a population of persons with a DSM-IV diagnosis of Substance Use Disorder who were court ordered for treatment. Two groups of adults (N = 40) were closely matched on age, gender, race, socioeconomic status and education level, and were administered the Person Picking…

Some probabilistic illustrations of the reliability coefficient are provided to assist in interpretation of this measure. All explanations are derived under the assumption that the joint distribution of examinee scores from two parallel tests is well approximated by a bivariate normal distribution.

The reliability of a taxonomy of humor was tested in two studies. The first study involved rater identification of nine categories for humorous incidents excerpted from television comedy programs (wordplay, exaggeration/understatement, contrast, audience knowledge, aggression, emotion, taboo, pratfall/slapstick, and repetition). The second study,…

This paper explores the use of K. G. Joreskog's (1970) congeneric modeling approach to reliability using censored quantitative variables. Two Monte Carlo studies were conducted. The first explored the robustness of Normal Theory Generalized Least-Squares (NTGLS) estimates for a single-factor congeneric model across several sample sizes…

The purpose of this study is to determine (1) whether the previously observed vulnerability of WISC subtests to tester effects appeared under ordinary testing conditions, and (2) which subtests were most susceptible to these effects. Results support the presence of both general and differential vulnerability of subtests. (Author)

This study was designed to: (1) assess the influence of an instructional set given to an observer regarding the presence or absence of a predictable pattern in the observed interaction, (2) extend and replicate findings of a previous study of observer accuracy, and (3) identify some of the specific types of errors made by observers in coding…