The authors used cross-cultural methodology to examine the demand-withdraw pattern of marital communication. In Western countries, women usually make more demands, whereas men are more likely to withdraw. But the recently advanced marital structure hypothesis suggests that this pattern can be altered by gender roles and beliefs, particularly in…

This is a construct validation study of a second language speaking assessment that reported a language profile based on analytic rating scales and a composite score. The study addressed three key issues: score dependability, convergent/discriminant validity of analytic rating scales and the weighting of analytic ratings in the composite score.…

This paper presents a review of the literature on reliability in qualitative studies. Reliability is defined as the extent to which studies can be replicated, using the same methods, and getting the same results. It is the degree to which data are independent of the accidental circumstances of the research. The review includes the following three…

Generalizability theory is based upon analysis of variance (ANOVA) and requires estimation of variance components for the ANOVA design under consideration in order to compute either G (Generalizability) or D (Decision) coefficients. Estimation of variance components has a number of alternative methods available using SAS, BMDP, and ad hoc…

The reliability, long-term stability, and generalizability of student ratings of teacher effectiveness are discussed. The Students' Evaluation of Educational Quality (SEEQ) instrument is examined from these perspectives. The multidimensionality of student response to such evaluation instruments must be recognized. (SLD)

This paper reports the results of generalizability theory (G) analyses done for new writing and speaking tasks for the Test of English as a Foreign Language (TOEFL). For writing, a special focus was placed on evaluating the impact on the reliability of the number of raters (or ratings) per essay (one or two) and the number of tasks (one, two, or…

The generalizability of scores from different scales in performance assessment was studied. First, a concept map of teachers' and raters' perceptions about various scores and scales was constructed using multidimensional scaling analysis. Then, a generalizability study using a random, partially nested design was conducted to analyze the…

With a construct-centered reliability analytical approach the reliability analysis should crystallize the multi-traits or constructs that the test specialists developed to measure from student performance and then estimate the degree of fit between the theoretical expectations of test developers and the performance exhibited by students. This…

Scholars from psychology and education describe how the new rules of measurement work and how they differ from the old rules. The collection provides an integrated and comprehensive reference source concerned with contemporary issues and approaches in testing and measurement. The chapters are: (1) "Issues in the Measurement of Cognitive…

Results and interpretations of the data from a performance examination were compared for four methods of analysis for 74 medical specialty certification candidates: (1) traditional summary statistics; (2) inter-judge correlations; (3) generalizability theory; and (4) the multifaceted Rasch model. Advantages of the Rasch model are outlined. (SLD)

Explains the use of the construct representation and inference functions of indicators used in construct measurements of self-concept. Illustrates validity and reliability issues with data from 2,114 12th-grade vocational-technical students. (SK)

Using results for 1,487 West German eleventh graders from the International Study of Achievement in Written Composition, problems related to interrater and intrarater reliability and generalizability of composition ratings were assessed. Results indicate that numerous writing assignments are required to assure reliability. (TJH)

A method is presented for determining the optimal number of conditions to use in multivariate-multifacet generalizability designs when resource constraints are imposed. A decision maker can determine the number of observations needed to obtain the largest possible generalizability coefficient. The procedure easily applies to the univariate case.…

A conceptual framework and heuristic model for considering the existence, magnitude, and consequences of context effects are presented through an extension of some generalizability theory concepts. Context effects are often misunderstood, and current measurement models have serious limitations for examining them. Their importance needs to be…

This study, which examined the number of minimal-word-pair exemplars necessary for 19 phonologically impaired children (ages 3-6) to meet a generalization criterion, found substantial variability across subjects, though generalization usually occurred following a small number of exemplars. No relationship was found between specific sounds and the…