A new measure for reliability of a rating scale is introduced, based on the classical definition of reliability, as the ratio of the true score variance and the total variance. Clinical trial data can be employed to estimate the reliability of the scale in use, whenever repeated measurements are taken. The reliability is estimated from the…

Sixty-six English-speaking postgraduate distance-education medical students completed the Learning Styles Questionnaire (LSQ: 40-item version). This was completed while attending a residential workshop at the beginning of the semester, and 44 of these students completed the same LSQ questionnaire 5 months later at the completion of the semester.…

Mackintosh and Bennett's [Mackintosh, N. J. and Bennett, E. S, (2005). ''What do Raven's Matrices measure? An analysis in terms of sex differences.'' Intelligence 33: 663-674.] study shows that males outperform females in some APM items but not in others, implicating that these items are measuring discriminable mental processes. The present…

In this rejoinder to John Chubb's reply to "Edison Is the Symptom, NCLB Is the Disease," the author argues that Edison offers feel-good measures without really solving any of the problem of schools in poverty. Defending his original argument, the author cites a RAND study that questions the results Chubb claims. The study indicates the…

The Quebec Adaptive Behavior Scale (QABS) is widely used in Quebec (Canada) to assess behavior of people with mental retardation in educational, vocational, residential or hospital settings. This study estimated the interrater agreement and test-retest reliability of the QABS. To determine test-retest reliability, the QABS was completed by 27…

The reliability and validity of teacher ratings, the relationship between teacher ratings and principal self-reports of instructional leadership, and the degree to which they are influenced by demographic factors are examined in this study. Methodology involved completion of the Instructional Leadership Inventory, a self-report measure, by 81…

This research aimed to investigate the effect of computers on the test and inter-rater reliability of writing test scores of ESL learners. Writing samples of 20 pen-paper and 20 computer group students were scored in analytic scoring method by two scorers, and then the scores were analyzed in Alpha (Cronbach) model. The results showed that the…

A total of 2,000 essays written by 1,000 students was submitted to generalizability analyses for domain-referenced tests. Each student had written one essay on each of two prompts representing two models of discourse. Each essay was read by six readers and judged on a scale of from 1 to 4. No reader read essays from both prompts. Reader agreement…

This report shows how Beta weights for the J-Coefficient may be easily developed without a formal validity study, and indicates how indications of ability other than tests can be used to measure the same abilities that are measured by tests. See also TM 001 163-64,166 for further information on job elements (J-Scale) procedures. (Author/DLG)

Perhaps more than ever before, college teaching is being studied and evaluated. This paper describes the development of a simple descriptive instrument used to focus observers' classifications and ratings of college teachers' instructional behaviors as recorded on video tape. The need for such an instrument is reviewed, the methodology for testing…

This study examined whether variations in the Developmental Observation Checklist (DC) format influences congruence of scores among both parents and the child's teacher. The DC was varied by adding pictorial illustrations and examples and having three response categories instead of two. Results from 100 sets of participants were evaluated with…

The Child Behavior Checklist and related forms were completed for 171 boys referred for school-based assessment resulting from academic and/or behavioral problems. Adolescents consistently underreported behavioral problems relative to parents and teachers regardless of subsequent diagnosis. Implications of these discrepancies in school-based…

To assess cross-observer agreement on personality profiles, an Index of Profile Agreement and an associated coefficient are proposed that take into account both the difference between the ratings and the extremes of their mean. Data from the Revised NEO Personality Inventory for 250 peer ratings/self-reports and 68 spouse ratings/self-reports…

Investigated the reliability and item appropriateness, as discerned by adults affiliated with an infant center, of the Caregiver-Environment Scale (CES). Found the CES to be an easy to use, reliable instrument for evaluation. (Author/SD)

This study explored the use of the Formal Elements Art Therapy Scale (FEATS) with a population of persons with a DSM-IV diagnosis of Substance Use Disorder who were court ordered for treatment. Two groups of adults (N = 40) were closely matched on age, gender, race, socioeconomic status and education level, and were administered the Person Picking…