Presents a scale to measure teacher sense of work autonomy with evidence for its score replicability. Results of replicability analyses (cross validation and validity generalization) involving 156 Israeli elementary school teachers and 650 Israeli elementary and secondary school teachers suggest four areas of functioning pertinent to teachers'…

Yen and Burket suggested that shrinkage in vertical equating cannot be understood apart from multidimensionality. Reviews research on reliability, multidimensionality, and scale shrinkage, and explores issues of practical importance to educators. (SLD)

This study examined interjudge agreement levels for five adult listeners assessing either overt stuttering or disfluency types in the spontaneous speech of eight young children. Results showed that the interjudge reliability for judgments based on a disfluency taxonomy was not significantly different from that based on stuttering. The importance…

Reexamines the reliability and validity of the Peabody Individual Achievement Test-Revised (PIAT-R) and the Wechsler Individual Achievement Test (WIAT) Written Expression scoring systems, applying theory and more appropriate statistical analyses. The WIAT written-expression scoring system was found to have more items that were both reliable and…

Reviews and integrates existing child and adolescent Hand Test research for use with school systems. Examines reliability and validity of the instrument along with moderating variables that impact interpretation. Favorably views the Hand Test as a brief protective technique that can easily be integrated into school psychologists' test battery.…

Proposes a method for obtaining standard errors and confidence intervals of composite reliability coefficients based on bootstrap methods and using a structural-equation-modeling framework for estimating the composite reliability of congeneric measures (T. Raykov, 1997). Demonstrates the approach with simulated data. (SLD)

Developed a video portfolio technique of teacher assessment and evaluated the technique through studies of six teachers and their raters. Results show that teachers are consistent in observing teaching functions and using their observations to evaluate teaching. (SLD)

Behavior measures and the Myers Briggs Type Indicator were completed by 343 senior managers; 3,158 of their peers, supervisees, and supervisors rated managers' behavior. A modest correlation appeared between personality type and manager behavior. Differences related to raters' perceptions were found. (SK)

This study examined the inter-rater reliability of clinician-assigned diagnosis of autism using or not using the criteria specified in the Diagnostic and Statistical Manual IV (DSM-IV). For experienced raters there was little difference in reliability in the two conditions. However, a clinically significant improvement in diagnostic reliability…

Suggests that a modification of traditional linking is necessary when tests consist of constructed response items judged by raters and a possibility of year-to-year variation in rating discrimination and severity exists. Illustrates this situation with an artificial example. (SLD)

Self, subordinate, peer, and supervisor ratings of 1,080 managers in education, military, government, manufacturing, finance, and health were analyzed for leniency, interrater agreement, and effectiveness. In the private sector, more poor performing managers tended to overestimate their performance. Interrater agreement was lowest in government…

Generalizability theory is described as a way to provide a unified framework for examining the dependability of measurements in the health sciences. Generalizability theory is appropriate whether the measurement is norm- or criterion-referenced, an individual or aggregate-level variable, and it is useful when there are single or multiple sources…

This study developed a testing approach to increase sample size to approximately 450 scorable items, previously determined as necessary to optimize the reliability of a speech recognition test. The approach involves an interactive computer program which presents words in 50 three-word groups. The approach was evaluated with 100 persons with normal…

Reviews relevant aspects of generalizability theory related to performance assessments and discusses the role of various facets in assessing the generalizability of performance assessments. Also considers some popular estimates of reliability for performance assessments from the perspective of generalizability theory. (SLD)

Investigated the reliability and validity of the Student Activities Interest Questionnaire, developed to relate student interests to available campus activities using Holland's (1997) vocational interest categories. Found the instrument to be sufficiently reliable and valid for use as an informal means of prioritizing campus activities. (EV)