Demonstrates that some commonly used indices can be misleading in their quantification of reliability. The effects are most pronounced on gain or difference scores. Proposals are made to avoid sources of invalidity by using a procedure to assess reliability in terms of upper and lower limits for the true scores of each examinee. (Author/JDH)

Provides information on statistical data and jargon so that English department members can more confidently and responsibly identify the inevitable weaknesses and limitations of both tests of writing ability and the research on them. (RL)

Since speech communication evaluators are beginning to adapt the analytic and holistic instruments and methods used for rating written products to oral products and performance, this research review investigated: (1) what the labels "analytic" and "holistic" mean; (2) the theoretical bases of the two scoring approaches; and (3)…

This volume is the fourth in a series for college faculty and advanced graduate students, "Survival Skills for Scholars." It offers practical advice for developing, using, and grading classroom examinations, focusing on traditional multiple-choice and constructed-response tests rather than alternative assessments. (SLD)

A general issue related to language program development involves the empirical rationalization of cut score decisions in criterion-referenced language tests. Cut score dependability focuses on the consistency of the decisions in repeated testing or the assessment of language learner performances. In this case, the issue is to determine the optimal…

A study investigated whether a positive correlation existed between scores obtained by incoming freshman on the recently developed Degrees of Reading Power Test (DRP) and the required Reading Comprehension subtest of the New Jersey College Basic Skills Placement Test (NJCBSPT). The subjects, 217 William Paterson College freshman enrolled in a…

Attitude measurement through Likert-type surveys usually provides no opportunity to assess the importance of the statements of the subjects. This study, involving 479 graduate and undergraduate students, examines the question of whether importance and agreement measures have different underlying dimensions, and examines the question of whether the…

The reliabilities of two types of measurement plans were compared across six hypothetical distributions of true scores or abilities. The measurement plans were: (1) fixed-length, where the number of trials for all examinees is set in advance; and (2) trials-to-criterion, where examinees must keep trying until they complete a given number of trials…

A measurement approach called generalizability theory (G-theory) is an important alternative to the more familiar classical measurement theory that yields less useful coefficients such as alpha or the KR-20 coefficient. G-theory is a theory about the dependability of behavioral measurements that allows the simultaneous estimation of multiple…

This paper outlines the development of a performance assessment measure of language speaking ability, the Language Ability Assessment System (LAAS), which is highly reliable and can be examined for reliability through modern measurement theories, such as generalizability theory (G-theory) and the many-facet Rasch theory. LAAS was developed to…

Scales constructed under procedures and criteria outlined by the various traditional and latent trait methods were examined as to whether they varied in characteristics related to scale quality. Scales were constructed from a common pool of items analyzed in full form according to Likert and a one-parameter Rasch model for non-dichotomous data.…

Investigates the utility and predictive validity of the Learning and Study Strategies Inventory (LASSI) as a means of measuring college students' cognitive and affective growth following a study strategies course. Finds cognitive and affective growth in both regularly admitted and developmental studies students. Finds that LASSI cannot yet be used…

Prior research in a single clinical training setting has shown Cason and Cason's (1981) simplified model of their performance rating theory can improve rating reliability and validity through statistical control of rater stringency error. Here, the model was applied to clinical performance ratings of 14 cohorts (about 250 students and 200 raters)…

This research study investigates the validity of the Teacher Locus of Control Scale (TLCS) in a different cultural environment. The scale, developed in the United States by Taylor et al., measures teachers' beliefs about their own potential to influence student performance and classroom events. Specifically, the study investigates the…

Items from two American instruments (Students' Evaluation of Educational Effectiveness, and the Endeavor instrument) designed to measure students' evaluations of teaching effectiveness were translated into Spanish and administered to a sample of Spanish university students. Most of the items were judged by the students to be appropriate; every…