Participants in achievement tests or psychometric scales can be naturally divided into various sub-groups such as gender, race, social economic status, school district, etc. In order to make meaningful comparison between groups, each item in the test/scale should measure the same underlying construct for participants came from different groups.…

Score equating based on small samples of examinees is often inaccurate for the examinee populations. We conducted a series of resampling studies to investigate the accuracy of five methods of equating in a common-item design. The methods were chained equipercentile equating of smoothed distributions, chained linear equating, chained mean equating,…

Measurement error is a common theme in classical measurement models used in testing and assessment. In classical measurement models, the definition of measurement error and the subsequent reliability coefficients differ on the basis of the test administration design. Internal consistency reliability specifies error due primarily to poor item…

Tests constructed through item-sampling (where several tests are constructed and randomly assigned to testees) were compared to a traditional constant-item procedure for evaluating student and class progress in a community college freshman psychology class. The item-sampling approach did not appear to be superior. (JKS)

Tests constructed through item-sampling (where several tests are constructed and randomly assigned to testees) were compared to a traditional constant-item procedure for evaluating class progress in a community college intermediate algebra class. The item-sampling format appeared to be slightly less valid than the traditional format. (JKS)

When the basic unit being studied is a grouping of individuals, both students and items can be sampled. When this method is applied to the use of log sheets to measure fidelity of program implementation, the normal limitations of self-reporting are reduced. (BW)

The use of matrix sampling to overcome the impracticality of the pair comparison method for obtaining scale values is empirically tested. Results indicate that matrix sampling is useful in such applications. (Author/JKS)

The assumption that an examinee's response to an item is independent of the context in which it occurs is tested. (PR)

A common approach to needs analysis involves the development of a self-report questionnaire administered to members of the intended audience. Such an approach poses a problem however: Concern for data reliability demands polling as many respondents as possible, while concern for cost-effectiveness dictates minimizing the size of the sample. The…

Test reliability is a concept central to classical test theory and it is commonly stated as a requirement that a test attain a certain level of reliability before it be considered of sufficient quality for practical use. This article discusses the role of reliability in item response theory, and in particular the role of reliability in contexts…

The effect of stratified sampling of items on the estimation of test score distribution parameters by multiple matrix sampling was studied. Item difficulty and/or interitem correlations were the bases of stratification. Various item iniverses were created by computer simulation and sampled according to several plans. The results indicate that…

These results indicate that in deciding on the data-gathering design to be used in seeking norm information, attention should be given to item characteristics and test length with particular attention paid to the range of biserial correlations between item response and ability. (Author)

In a project to identify and verify professional competencies for beginning industrial education teachers, researchers found a 173-item questionnaire unwieldy. Using multiple-matrix sampling, they distributed subsets of items to respondents, resulting in adequate returns as well as duplication, postage, and time savings. (SK)

Describes the evolution of national educational assessment in the United States, its present stage of development, methodological and reporting problems, and approaches applicable to assessment data. Examines how proposed approaches can be applied in the California Assessment Program and in the National Assessment of Educational Progress. (MJL)

The net effect of the conditions under which tests are taken was empirically investigated using the scores obtained by high school students on an English and a mathematics test. (Author/BB)