The limitations of Cohen's ? are reviewed and an alternative G-index is recommended for assessing nominal-scale agreement. Maximum likelihood estimates, standard errors, and confidence intervals for a two-rater G-index are derived for one-group and two-group designs. A new G-index of agreement for multirater designs is proposed. Statistical…

Different sources of assessment evidence are reviewed during International Baccalaureate (IB) grade awarding to convert marks into grades and ensure fair results for students. Qualitative and quantitative evidence are analysed to determine grade boundaries, with statistical evidence weighed against examiner judgement and teachers' feedback on…

We propose a coefficient of agreement to assess the degree of concordance between two independent groups of raters classifying items on a nominal scale. This coefficient, defined on a population-based model, extends the classical Cohen's kappa coefficient for quantifying agreement between two raters. Weighted and intraclass versions of the…

This guide is designed to help researchers make sure that their research reports include enough information about study measures so that readers can assess the quality of the study's methods and results. The guide also provides examples of write-ups about measures and suggests resources for learning more about these topics. The guide assumes…

A recent series of papers has renewed interest in the question of whether consumption data are superior to income data for poverty measurement. Although the Census Bureau has provided researchers with an experimental series of variables that can produce a comprehensive income measure, this resource has not been fully exploited in previous…

Agreement between two independent reviews of each of 278 manuscripts was compared on an overall recommendation and on specific rating items. Agreement between reviewers on separate dimensions, the unweighted sum of the dimensions, and various weighted sums was no better than that for the overall recommendation itself. (SLD)

Investigated the relationship between Chinese character recognition and production by second-language learners. Subjects were 47 first-year Chinese language students in the United States. (15 references) (Author/CK)

Understanding that the behavior of people takes place within a context, over the past 20 years research in education and the sport sciences has witnessed an increasing development of multilevel frameworks that are both conceptually and methodologically sound. Despite these advances, the use of multilevel models and research designs in education…