The aptitude tests used in Queensland (Australia) to standardize secondary school leavers' school evaluations was assessed for reliability with various student group sizes and for a range of abilities. The error found was significant for small and moderately sized groups, suggesting the need for care in test interpretation. (Author/MSE)

Although higher education institutions dislike rankings published in the mass media, they like the attention the rankings create and prefer to be included rather than excluded. Common criticisms of the methodology include emphasis on inappropriate criteria, unfair comparison of private and public institutions, faulty assumptions, inaccurate data,…

Investigates the reliability of the Instructional Activity Questionnaire, Form 3, Revised, in measuring instructional leadership behaviors of special education supervisors, using a sample of 38 supervisors and 133 special education teachers. Using Cronbach's coefficient alpha, acceptable reliability is found for special education supervisors. (SLD)

This study investigated two pragmatic profiles (the Pragmatic Profile and the Profile of Communicative Appropriateness) used to assess the language of two aphasic patients. The study examined interscorer reliability, scoring sensitivity, and diagnostic accuracy. Findings indicate that training in scoring these profiles must be uniform, and greater…

Two studies investigated interrater agreement among 10 clinical dental examiners who scored residual subgingival calculus after student scaling on 4,160 real and 92 manikin tooth surfaces. Interrater reliability was low. Results suggest a need in periodontics for effective examiner calibration methods and objective subgingival calculus detection…

The Wechsler Individual Achievement Test assesses achievement in students in grades K-12, with subtests in reading, mathematics reasoning, spelling, reading comprehension, numerical operations, listening comprehension, oral expression, and written expression. The test was developed to assist in identification of students with learning…

Dependability of the Children's Depression Inventory (CDI) was studied using both generalizability and classical test score analyses with a sample of 164 elementary school students. Results suggest that sources of error variance interact to decrease dependability of CDI scores. Depression in children might be better assessed through multiple…

The place of peer review in the process by which the Australian Research Council allocates funds under the Large Grants Scheme, the primary funding source for basic research, is examined. Validity and reliability of peer review, the decision-making process, and cost effectiveness of the procedures are considered. (MSE)

Two studies: evaluated the consistency of toddlers' motor activity level over time and across situations; and examined the relations between several measures of activity level and ratings of problem behavior in toddlers. Found that activity level was stable when measured by the same methods in the same situation but not across methods or across…

The Developmental Activities Screening Inventory II (DASI-II) was administered to a group of 13 toddlers with severe disabilities. Results indicated that interrater reliability was statistically significant across raters for all scores reported by the DASI-II and that test-retest stability was statistically significant across administrations and…

A method is proposed for sequential evaluation of reliability of psychometric instruments. Sample size is unfixed; a test statistic is computed after each person is sampled and a decision is made in each stage of the sampling process. Results from a series of Monte-Carlo experiments establish the method's efficiency. (SLD)

After 98 lecture and 43 course evaluations, rating scales used by students to evaluate faculty at Victoria University of Wellington (New Zealand) were subjected to factor analysis. It was found that pooling ratings from different teaching situations made possible preparation of a teaching performance profile improving the information's reliability…

Possible sources of intrajudge inconsistency in standard setting are reviewed, and approaches are presented to improve the accuracy of rating. Procedures for providing judges with feedback through discussion or computerized communication are discussed. Monitoring and maintaining judges' consistency throughout the rating process are essential. (SLD)

Two frame-of-reference rater training approaches were compared for effects on reliability and accuracy of cutoff scores generated by 21 raters using Angoff methods on tests taken by 155 undergraduates. Both approaches result in higher interrater reliability and more accuracy than does a non-frame-of-reference method. (SLD)

Parent and teacher ratings on the Aberrant Behavior Checklist with an outpatient sample of 110 children, adolescents, and young adults with mental retardation found that the 5-factor structure of both parent and teacher data corresponded very well with the 5 factors originally obtained from staff ratings of mentally retarded inpatients. (Author/DB)