A more general method for calculating the Kappa measure of nominal rating agreement among multiple raters is presented. It can be used across a broad range of rating designs, including those in which raters vary with respect to their base rates and how many subjects they rate in common. (Author/BW)

Evaluating a test's reliability often requires dividing it into 3 or more unequal parts, which causes violation of the tau equivalence assumption of Cronbach's alpha. This article presents a criterion for abandoning alpha and an approach for computing a more appropriate estimate of reliability, the Gilmer-Feldt coefficient. (Author)

Examined the reliability and validity of a career commitment measure using employees (N=266) of newspaper and insurance companies. Results showed career commitment could be reliably measured and was operationally distinct from job involvement and organizational commitment. Discusses findings in terms of meaning of career commitment. (Author/ABL)

Gulliksen's matched random subtests method is a graphical method to split a test into parallel test halves, allowing maximization of coefficient alpha as a lower bound to the classical test reliability coefficient. This problem is formulated as a zero-one programing problem solvable by algorithms that already exist. (TJH)

The idea of test consistency is illustrated, with reference to two sets of test scores. A mathematical model is used to explain the relative consistency and relative inconsistency of measurements, and a means of indexing reliability is derived using the model. Practical aspects of estimating reliability are considered. (TJH)

We make theoretical comparisons among five coefficients--Cronbach's [alpha], Revelle's [beta], McDonald's [omega][sub h], and two alternative conceptualizations of reliability. Though many end users and psychometricians alike may not distinguish among these five coefficients, we demonstrate formally their nonequivalence. Specifically, whereas…

This paper reports on the development of a scale for determining the quality of the student e-learning experience at the degree level when the student learning context is predominately a campus-based experience. Rapid developments in the use of information and communication technologies (ICT) in higher education require methods for evaluating the…

Due to the relative inability of individuals with intellectual disabilities (ID) to provide an accurate and reliable self-report, assessment in this population is more difficult than with individuals in the general population. As such, assessment procedures must be adjusted to compensate for the relative lack of information that the individual can…

Career and Technical Education (CTE) is a nationwide program that emphasizes training for primary, secondary, and post secondary educational stages for the career and workforce needs of today and tomorrow's society. Mandated indicators of success have been set in place and secondary schools are expected to improve student's skill levels in…

In the field of educational psychology, there is diverse and active research in motivation for learning and achievement. Many instruments exist for assessing students' motivation, primarily as self-report. Fewer instruments are available for assessing "teachers'" perceptions of their students' motivation, and fewer still for assessing teachers'…

Accurate measures of youth fitness are needed by researchers and practitioners. Evidence of validity and reliability are essential before results of youth fitness tests can be used to make sound decisions. This article describes a three-stage paradigm for validation research and provides guidance for conducting and understanding norm-referenced…

Two studies examined the psychometric properties of the Posttraumatic Stress Disorder (PTSD) subscale of the SCL-90-R. Study 1 examined SCL-90-R responses from 2,361 college women to determine whether this subscale can appropriately assess the three dimensions of PTSD. Factor analysis and Cronbach's alpha suggest that this subscale is best…

Student evaluations of advising (SEA) are often limited to student ratings of the faculty member's advising skills. As with teaching evaluations, ratings of SEA may not be the best reflection of the advisor's performance. In this study, the author analyzed SEA to determine the relationship between student characteristics and evaluation items.…

This article documents the development and use of a survey instrument designed to measure K-8 mathematics teachers' perceptions about discourse in mathematics classes. In particular, the 5-point Likert-type survey sought to address teachers' perceptions of their use of dialogic (dialogue to construct new meaning), univocal (conveying information),…

This study extends prior work with the Russian-language version of the Demands of Immigration Scale (DIS) with former Soviet immigrants and evaluates the reliability and validity of the Arabic-language version of the DIS with Arab immigrants. Three hundred and ninety-four Arab immigrant women completed the Arabic DIS and two measures of mood.…