Reliability is a property of test scores from individuals who have been sampled from a well-defined population. Reliability indices, such as coefficient and related formulas for internal consistency reliability (KR-20, Hoyt's reliability), yield lower bound reliability estimates when (a) subjects have been sampled from a single population and when…

The authors introduce a consistency-based approach to detecting examiner bias. On comparing intra-class correlation coefficients on transformed data for supervisor continuous performance and report marks (ICC1*) with those for supervisor continuous performance and second marker report marks (ICC2*), a highly significant difference was obtained…

Purpose: This article provides information about clinical practice guidelines (CPGs) to facilitate their application to the practice of speech-language pathology. CPGs are sets of recommendations based on evidence, including expert clinical opinion, that have been developed by a panel of reviewers. In this article, CPGs are defined and their…

The Five-Factor Personality Inventory (FFPI) assesses a person's position on the (Dutch) psycholexically based Big Five factors: Extraversion, Agreeableness, Conscientiousness, Emotional Stability, and Autonomy. FFPI factor scores are reliable and valid if ratings are made by adults. The present study yields preliminary evidence of whether young…

This article examines the reliability of content analyses of state student achievement tests and state content standards. We use data from two states in three grades in mathematics and English language arts and reading to explore differences by state, content area, grade level, and document type. Using a generalizability framework, we find that…

The latent state-trait (LST) theory is an extension of the classical test theory that allows one to decompose a test score into a true trait, a true state residual, and an error component. For practical applications, the variances of these latent variables may be estimated with standard methods of structural equation modeling (SEM). These…

The consensual assessment technique (CAT) is a measurement tool for creativity research in which appropriate experts evaluate creative products [Amabile, T. M. (1996). "Creativity in context: Update to the social psychology of creativity." Boulder, CO: Westview]. However, the CAT is hampered by the time-consuming nature of the products (asking…

Reliability consists of both important social and scientific values and methods for evidencing those values, though in practice methods are often conflated with the values. With the two distinctly understood, a reliability argument can be made that articulates the particular reliability values most relevant to the particular measurement situation…

Effective use of reading strategies has been recognized as an important means to increase reading comprehension. Many English as a Foreign Language (EFL) or English as a Second Language (ESL) studies have produced lists of paper-reading strategies (e.g. having a purpose for reading; using context clues). In contrast, few studies have investigated…

The No Child Left Behind Act of 2001 (NCLB, 2002) has produced an explosion of interest in the use of assessment to measure and improve student learning. Initially focused on annual state tests, educators quickly learned that results came too little and too late to identify students who were falling behind. At the same time, evidence from the…

The use of accommodations for both instruction and assessment continues to be of great importance for students with disabilities. The purpose of this report is to provide an update on the state of the research on testing accommodations, as well as to identify promising areas of research likely to contribute to understanding of current and emerging…

The benefits of using data logging to teach "how science works" are presented. Pedagogical approaches that take advantage of other school ICT are briefly described. A series of simple, quick experiments are given together with their resulting charts. Examples of the questions that arise from the charts show how the rich data lead to the refinement…

This study examined the content of email messages to the listserv "CESNET-L" in order to identify trends, common themes, and "hot topics;" to clarify its purpose; and to offer suggestions for the future of CESNET-L and similar email lists in higher education. CESNET-L is an unmoderated listserv mostly used by counselor educators and doctoral…

The use of standardized tests for outcome assessment has grown dramatically in recent years. Two driving factors have been the No Child Left Behind legislation, and the increase in outcome assessment measures by accrediting agencies such as AACSB, the international accrediting body for business schools. Despite the growth in usage, little effort…

Background: Cultural competency is an important but under-adopted skill among professional evaluators. Yet in the transactions around job seeking and hiring in evaluation, cultural competency is a practical and common concept. How cultural competency gets communicated in those transactions may provide insights for the field. Purpose: The purpose…