Multivariate methods are being used with increasing frequency in educational research because these methods control "experimentwise" error rate inflation, and because the methods best honor the nature of the reality to which the researcher wishes to generalize. This paper: explains the basic logic of canonical analysis; illustrates that…

The power of a statistical test is, in part, a function of the reliability of the dependable variable being analyzed. The substitution of sigma square divided by the reliability coefficient for sigma is proposed. This enables the researcher to incorporate dependent variable reliability information when determining the sample size required for a…

An approximate statistical test is derived for the hypothesis that the intraclass reliability coefficients associated with two measurement procedures are equal. Control of Type 1 error is investigated by comparing empirical sampling distributions of the test statistic with its derived theoretical distribution. A numerical illustration is…

These results indicate that in deciding on the data-gathering design to be used in seeking norm information, attention should be given to item characteristics and test length with particular attention paid to the range of biserial correlations between item response and ability. (Author)

A general framework for conducting generalizability analyses is presented. Generalizability theory is extended to situations in which the objects of measurement are not persons but other factors, such as instructional objectives, stages of learning, and treatments. (Author/PN)

Least squares fitting process as a method of data reduction is presented. The general strategy is to consider fitting (linear) models as partitioning data into a fit and residuals. The fit can be parsimoniously represented by a summary of the data. A fit is considered adequate if the residuals are small enough so that manipulating their signs and…

A set procedure developed in this study is useful in determining sample size, based on specification of linear contrasts involving certain formula treatments. (Author/DEP)

An experimental study of the efficiency of human information processing is based on the Bayesian model for simple hypothesis testing with fixed binomial sampling. Each of 60 subjects is analyzed with separate ANOVAs focusing on two efficiency variables. Sample size and critical value are also analyzed. Subjects show very different utilization of…

Results of this study suggest that certain kinds of treatment differences may be better uncovered by viewing test data in a repeated measures format. (Author/MB)

This report gives some simple examples of stability for one factor and 2 x 2 factorial analysis of variance, reliability and correlations. The findings are very different: from superstability (no transformation whatsoever can change the result) to almost total instability. This is followed by a discussion of applications to multivariate analysis,…

Multiple matrix sampling (MMS) theory indicates how data may be gathered to most efficiently convey information about levels of attainment in a population, but standard analyses of these data require random sampling of items from a fixed pool of items. This assumption proscribes the retirement of flawed or obsolete items from the pool as well as…

In the Spring, 1996, issue of "CRESST Line," E. Baker and R. Linn commented that, in efforts to measure the progress of schools, "the fluctuations due to differences in the students themselves could conceal differences in instructional effects." This is particularly true in the context of the evaluation of adequate yearly…

Describes advances in capture-recapture modeling, including the biologically motivated emphasis on model-based hypothesis testing and the development of models for spatially stratified populations. Discusses the severe problems associated with count statistics reflecting unknown sampling fractions. (67 references) (KR)

Evidence is presented on the generalizability and convergent validity of performance assessments using data from six studies of student achievement that sampled a wide range of measurement facets and methods. Results at individual and school levels indicate that task-sampling variability is the major source of measurement error. (SLD)

Rules, procedures, and algorithms intended to aid researchers and practitioners in the application of generalizability theory to a broad range of measurement problems are presented. Two examples of measurement research are G studies, which examine the dependability of some general measurement procedure; and D studies, which provide the data for…