The administrative and political ramifications of a form of multiple matrix sampling where all students in the population are tested on a subtest is discussed in the context of a statewide testing program. The conclusion is drawn that large-scale testing programs will have to adopt a testing framework which samples items from the total item domain…

This investigation had two major purposes. The first was to explore the use of an inferential technique called Tukey's Jackknife in establishing a confidence interval about cooefficient alpha reliability. The second purpose was to study the robustness of the Feldt and the jackknife procedures when the data fails to satisfy usual normality…

A common approach to needs analysis involves the development of a self-report questionnaire administered to members of the intended audience. Such an approach poses a problem however: Concern for data reliability demands polling as many respondents as possible, while concern for cost-effectiveness dictates minimizing the size of the sample. The…

A historical perspective on and substantive review of the concept of invariance are provided. Progress made toward solving measurement problems related to invariance is also assessed. Two major classes of invariant measurement are described: (1) sample-invariant item calibration; and (2) item-invariant measurement of individuals. The work of S. S.…

This study involving 352 students was designed to verify empirically the a priori use of multiple matrix sampling procedures in an elementary school using a nationally normed, commercizlly published achievement test. The study focused on effect of changes in item context, effect of previous exposure to items, and relative effectiveness of multiple…

The Michigan Oral Language program was installed in the Santa Paula School District for a 2-month tryout and student performance before and after the program was assessed through multiple matrix sampling. Student performance over seven content areas was determined for both the English and Spanish language components. In all seven content areas,…

Demands for more complete information on educational programs have emanated from national, state and local sources. Their focus is on the processes that are occurring in individual classrooms. The information that is collected to provide insight into educational programs is customarily summative in nature, answering, for example, questions…

In a situation where the population distribution of latent trait scores can be estimated, the ordinary maximum likelihood estimator of latent trait scores may be improved upon by taking the estimated population distribution into account. In this paper empirical Bayes estimators are compared with the liklihood estimator for three samples of 300…

A comparison was made between four different scoring procedures for the 45-item Reading Comprehension subtest, Level I, of the Comprehensive Test of Basic Skills, Form S, for a sample of 1,000 third grade students. These students were selected to have been among those who omitted from 3 to 22 of the 45 items. Another representative sample of 1,300…

The adequacy of a test developed for statewide assessment of basic mathematics skills was investigated. The test, comprised of multiple-choice items reflecting a series of behavioral objectives, was compared with a more extensive criterion measure generated from the same objectives by the application of a strict item sampling model. In many…

Multiple matrix sampling (MMS) theory indicates how data may be gathered to most efficiently convey information about levels of attainment in a population, but standard analyses of these data require random sampling of items from a fixed pool of items. This assumption proscribes the retirement of flawed or obsolete items from the pool as well as…

A test theory using only ordinal assumptions is presented, based on the idea that the test items are a sample from a universe of items. The sum across items of the ordinal relations for a pair of persons on the universe items is analogous to a true score. (SLD)

During the last 2 years (1980-82), selected schools in the Broward County School District in Florida participated in the National Sex Equity Demonstration Project (NSEDP) to create a model for demonstration of curricular materials, educational practices, and program arrangements that feature gender-fair instruction and associated educational…

In discussion of assessment and its applicability to the needs of the Spanish-speaking student population, four types of assessments are presented in the context of the educationally significant testing needed in this area. The focus of this paper is on the use of comprehensive assessment to measure the repertoire of language resources that the…

Multiple matrix sampling is a sampling design in which both test items and examinees are randomly sampled from their respective populations. This study was designed to develop and assess a method for computing an estimate of a correlation coefficient when a multiple matrix sampling design is used. The examinee populations included 212 third-grade…