The Collis Attitudes toward Computers Survey, developed by B. Collis (1984) for a secondary school population, was tested with 305 college students attending the University of Winnipeg (Canada). Results support the reliability and generalizability of the survey and identified one dominant factor--personal interest and enjoyment of computers. (SLD)

Dependability of the Children's Depression Inventory (CDI) was studied using both generalizability and classical test score analyses with a sample of 164 elementary school students. Results suggest that sources of error variance interact to decrease dependability of CDI scores. Depression in children might be better assessed through multiple…

Two frame-of-reference rater training approaches were compared for effects on reliability and accuracy of cutoff scores generated by 21 raters using Angoff methods on tests taken by 155 undergraduates. Both approaches result in higher interrater reliability and more accuracy than does a non-frame-of-reference method. (SLD)

Studied teacher certainty in 1,153 Norwegian teachers in elementary and junior high schools using confirmatory factor analysis and generalizability theory. Results indicate a good fit for a model that operationalizes teacher certainty as a second order latent variable with three first order latent variables: teacher's perceived didactic certainty,…

Purpose: To examine the validity of using scores from a clinical evaluation form as an assessment of clinical competence. Method: Investigators collected a longitudinal clinical skills assessment database that included scores reflecting performance on standardized patient interactions, case-based learning performance, scores on multiple-choice…

A pilot study was conducted to evaluate and improve the rating procedure proposed for use in a research effort designed to assess the essay writing ability of college sophomores. Generalizability theory and the Many-Facet Rasch Model were each used to (a) estimate potential sources of error in the rating, (b) to obtain reliability estimates, and…

Formulating-Hypotheses (F-H) items present a situation and ask the examinee to generate as many explanations for it as possible. This study examined the generalizability, validity, and examinee perceptions of a computer-delivered version of the task. Eight F-H questions were administered to 192 graduate students. Half of the items restricted…

Recently, there has been a significant expansion in the use of performance assessment in large scale testing programs. Although there has been significant support from curriculum and policy stakeholders, the technical feasibility of large scale performance assessments has remained a question. This report is intended to contribute to the debate by…

Data from performance measures that were part of two foreign language teacher certification examinations were used in a generalizability study of the quality of their performance ratings. A total of 775 examinees from the Spanish K-12 and 192 examinees from the French K-12 subject area tests of the Florida Teacher Certification Examinations were…

This paper argues that special procedures for constructing assessment tools containing performance assessment tasks are unnecessary and that current test methodology can easily be generalized to complex performance assessment tasks without destroying the desirable characteristics of those tasks. Reasonable statistical requirements for sound…

Most training programs in education and psychology focus on classical test theory techniques for assessing score dependability. This paper discusses generalizability theory and explores its concepts using a small heuristic data set. Generalizability theory subsumes and extends classical test score theory. It is able to estimate the magnitude of…

Classroom researchers are frequently urged to provide evidence of the reliability of their data. In the case of observational data, three approaches to this have emerged: observer agreement, generalizability theory, and measurement error. Generalizability theory provides the most powerful approach given an adequate data collection design, but…

The purpose of this study was to provide evidence for adapting and generalizing the use of the Children's Academic Motivation Inventory (CAMI) to high school students. The instrument was originally developed to provide a reliable, valid, theory-based measure of academic achievement motivation; it was suitable for use with children aged 12-14…

The application of generalizability theory to the reliability and error variance estimation for performance assessment scores is discussed. Decision makers concerned with performance assessment need to realize the restrictions that limit generalizability such as limitations that lead to reductions in the number of tasks possible, rater quality,…

Issues of dimensionality, validity, reliability, and generalizability of students' ratings of instruction were studied with 610 college students who evaluated their instructors on the Teacher Effectiveness Scale. The Scale had high internal consistency, but interrater reliability, generalizability, and predictive validity warrant further study.…