Many well-known equations in classical test theory are mathematical identities in populations of individuals but not in random samples from those populations. First, test scores are subject to the same sampling error that is familiar in statistical estimation and hypothesis testing. Second, the assumptions made in derivation of formulas in test…

In large-scale assessments, such as the National Assessment of Educational Progress (NAEP), plausible values based on Multiple Imputations (MI) have been used to estimate population characteristics for latent constructs under complex sample designs. Mislevy (1991) derived a closed-form analytic solution for a fixed-effect model in creating…

A jack knife is a pocket knife that is put to many tasks, because it's ready to hand. Often there could be a better tool for the job, such as a screwdriver, a scraper, or a can-opener, but these are not usually pocket items. In statistical terms, the expression implies making do with what's available. Another simile, of an extreme situation, is…

Purpose: The main aim of this paper is to develop guidelines on the critical issues to consider in research design in an action research (AR) environment for SME network capability development. Design/methodology/approach: The issues in research design for AR studies are developed from the authors' experience in running learning sets but, in…

A common problem in predictive validity studies in the educational and psychological fields, e.g. in educational and employment selection, is restriction in range of the predictor variables. There are several methods for correcting correlations for restriction of range. The aim of this paper was to examine the usefulness of two approaches to…

The development of a nonparametric K-sample test for equality of slopes using Puri's generalized L statistic is presented. The test is recommended when the assumptions underlying the parametric model are violated. This procedure replaces original data with either ranks (for data with heavy tails) or normal scores (for data with light tails).…

A computer program randomly sampled ordered pairs of scores from known populations that departed from bivariate normal form and calculated correlation coefficients from sample values. Hypotheses were tested (1) that population correlations are zero using the t statistic; and (2) that population correlations have non-zero values using the r to z…

The problem of predicting universe scores for samples of examinees based on their responses to samples of items is treated. The measurement model categorizes items according to the cells of a table of test specifications, and the linear function derived for minimizing error variance in prediction uses responses to these categories. (Author/JKS)

Equating a new standard test to an old reference test is considered when samples for equating are not randomly selected from the target population of test takers, identifying two problems from equating from biased samples. An empirical example with data from the Armed Services Vocational Aptitude Battery illustrates the approach. (SLD)

Linear conversions were developed relating scores on recent forms of the Graduate Record Examinations. Conversions based on specially selected subpopulations were compared with total-group conversions and evaluated. Conclusions indicated that the data clearly support the assumption of population independence for homogenoeous tests, but not quite…

The reliability and validity of measurement is analyzed by a sampling model based on generalizability theory. A model for the relationship between a measurement procedure and an attribute is developed from an analysis of how measurements are used and interpreted in science. The model provides a basis for analyzing the concept of an error of…

This paper discusses distribution theory and power computations for four common "tests of combined significance." These tests are calculated using one-sided sample probabilities or p values from independent studies (or hypothesis tests), and provide an overall significance level for the series of results. Noncentral asymptotic sampling…

A content analysis of 20 student conduct codes identified through stratified random sampling was performed to determine the extent to which legal terminology had been eliminated, as suggested by judicial affairs experts. The results showed that 80% of the codes selected in this study still contained some legal terms. These words and phrases are…

Standards, passing scores, or cut scores have been seen as an element of criterion-referenced tests since their introduction. This paper discusses at least two issues surrounding the establishment of cut scores which appear to need clarification: (1) the theoretical definition of a cut score; and (2) decisions which must be made in selecting a…

Data are summarized in Scheuneman's Score x Group x Response frequency table in order to investigate item bias. The data can arise from two different sampling models: (1) multinomial sampling in which a fixed sample size is used and the responses are cross-classified according to score, group, and response; and (2) product-multinomial sampling in…