Inter-rater reliability (IRR), which is a prerequisite of high-quality ratings and assessments, may be affected by contextual variables, such as the rater's or ratee's gender, major, or experience. Identification of such heterogeneity sources in IRR is important for the implementation of policies with the potential to decrease measurement error…

A procedure was developed to analyze data with missing observations by extracting data from a sparsely filled data matrix into analyzable smaller subsets of data. This subdividing method, based on the conceptual framework of meta-analysis, was accomplished by creating data sets that exhibit structural designs and then pooling variance components…

This study examined halo in raters' beliefs of item (attribute) relatedness. College students' prior beliefs of the co-occurrence of teaching attributes were correlated with actual correlation of teaching attributes of fictional college professors. Results showed some support for beliefs-of-relatedness source of halo. (LMO)