Many methods are used to measure interrater reliability for studies where each target receives ratings by a different set of judges. The purpose of this study is to explore the use of hierarchical modeling for estimating interrater reliability using the intraclass correlation coefficient. This study provides a description of how the ICC can be estimated using hierarchical modeling, recommends an appropriate non-parametric bootstrapping method, illustrates how both can be implemented to obtain a point estimate and an estimate of bias, and explores the viability of using these statistical tools to obtain such estimates. Results indicated that hierarchical modeling and the non-parametric bootstrap method can be used on both continuous and binary data to provide point and bias estimates of interrater reliability. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]