The consensual assessment technique (CAT) is a measurement tool for creativity research in which appropriate experts evaluate creative products [Amabile, T. M. (1996). "Creativity in context: Update to the social psychology of creativity." Boulder, CO: Westview]. However, the CAT is hampered by the time-consuming nature of the products (asking…

The "Special Ability Selection Examination" (SASE), which is used to select appropriate students for the music education departments of educational faculties in Turkey, has many subsections and must evaluate highly competitive cohorts of students according to a broad range of criteria. The test consists of three subsections, with a large…

Examined the factor structure of the Wechsler Adult Intelligence Scale-Revised (WAIS-R) in a sample of 200 medical patients and 271 psychiatric patients. Results indicate the WAIS-R has a robust factor structure and provide evidence of Verbal, Performance, and Freedom From Distractability factors in psychiatric and medical populations. (Author/BH)

In the first part of this paper we discuss the feasibility of using Generalizability (G) Theory to examine the dependability of concept map assessments and to design a concept map assessment for a particular practical application. In the second part, we apply G theory to compare the technical qualities of two frequently used mapping techniques:…

Much of the current rhetoric in areas of child and family research and in early childhood education emphasizes the importance of listening to children in research that has a direct impact on them. Despite this, there remain qualms in some research contexts and amongst some researchers about the reliability, validity and generalizability of…

This paper describes how work by the United States Office of Personnel Management on the generalizability of employment test validities led to the development of a widely applicable meta-analysis method. The method focuses strongly on estimating the true variance of study correlations and effect size. This validity generalization procedure has…

Using data from an individually administered interview schedule (the Consumer Satisfaction Inventory), reliability among nine interviewers was estimated with several statistical methods, including simple percentages of agreement, kappa and weighted kappa, Pearson correlations, t tests on interviewers' means, and generalizability theory techniques.…

Four approaches to estimating interrater reliability in early childhood special education research are illustrated and compared: correlation, comparison of means, percentage of agreement, and generalizability theory techniques. Generalizability theory techniques are proposed as a method for estimating the amount of variance attributable to…

This study used survival analysis to overcome problems of principled generalization and individual variability in analysis of the conversational speech of 24 children with speech delay recorded over two years. The derived normalization probabilities were lagged according to the strong delay hypothesis and results converged with previous normative…

Rating scales are a typical method for evaluating a student's performance in outcomes assessment. The analysis of the quality of information from rating scales poses special measurement problems when researchers work with faculty in their development. Generalizability measurement theory offers a set of techniques for estimating errors or…

In behavior research using cognitive and affective measures, there is often incongruity between the statistical analysis employed and the intended inference. This paper argues that incorporating items as levels of a random facet via generalizability theory allows the statistical examination of the inferential question in the desired universe of…

Classroom researchers are frequently urged to provide evidence of the reliability of their data. In the case of observational data, three approaches to this have emerged: observer agreement, generalizability theory, and measurement error. Generalizability theory provides the most powerful approach given an adequate data collection design, but…

Psychometric theory has been one of psychology's stronger foundations and a major contributor to the recognition of psychology as a scientific discipline. Basic principles of psychometric theory led to the development of respected intelligence tests and large and comprehensive testing and assessment programs. This paper synthesizes major…

The Stages of Concerns Questionnaire (SoCQ) is an established instrument which has been used primarily with inservice teachers involved in innovations. While it focuses on inservice teachers' concerns about an innovation, the instrument is based on theoretical constructs developed by the work of Frances Fuller and others during the 1960s with…

This study used generalizability theory and other statistical concepts to assess the application of the Angoff method to setting cutoff scores on two professional certification tests. A panel of ten judges gave pre- and post-feedback Angoff probability ratings of items of two forms of a professional certification test, and another panel of nine…