The "PISA 2009 Technical Report" describes the methodology underlying the PISA 2009 survey. It examines additional features related to the implementation of the project at a level of detail that allows researchers to understand and replicate its analyses. The reader will find a wealth of information on the test and sample design,…

The choice of design parameters (e.g., number of subtests, number of items per subtest, and number of examinees per subtest) can be controlled by the test constructor in a multiple matrix sampling evaluation. The purpose of this study was to determine empirically which combination of the above parameters produces the smallest standard errors of…

To teach the rigorous skills and knowledge students need to succeed in future college-entry courses and workforce training programs, education stakeholders have increasingly called for more rigorous curricula, instruction, and assessments. Identifying the critical attributes of rigor and measuring its appearance in curricular materials is…

The administrative and political ramifications of a form of multiple matrix sampling where all students in the population are tested on a subtest is discussed in the context of a statewide testing program. The conclusion is drawn that large-scale testing programs will have to adopt a testing framework which samples items from the total item domain…

When performing large scale evaluations (e.g., on a state-wide or national level) it may not be possible to administer all items in the item universe to all respondents in the subject population. One method which has been proposed to sample both items and respondents is multiple matrix sampling (MMS) in which a sample of the items is administered…

Shoemaker's conclusions related to the influence of various data base characteristics (reliability, variability of item difficulty indices, and degree of skewness in the normative distribution) on the standard error of a mean estimated via multiple matrix sampling procedures are examined. (Author/RC)

Generalized symmetric means are redefined in a way which allows them to be calculated for any matrix sampling design. It is proved that these sample generalized symmetric means are unbiased estimates of the analogous population generalized symmetric means. Illustrative examples are given. (Author)

This note refers to EJ 056 482. (CB)

As a result of the federal No Child Left Behind Act (NCLB) of 2002, the field of education has seen a heavy emphasis on the use of "scientifically based research" for designing and testing the effectiveness of new and existing educational programs. According to NCLB, when addressing basic cause and effect questions scientifically based…

Measurement error is a common theme in classical measurement models used in testing and assessment. In classical measurement models, the definition of measurement error and the subsequent reliability coefficients differ on the basis of the test administration design. Internal consistency reliability specifies error due primarily to poor item…

Internal consistency as one rationale for item selection from the unverse of possible test items is discussed and formulae are presented which relate the maximum internal consistency of a test to the largest eigenvalue of the interitem correlation matrix. A computer program to perform these calculations is presented. (Author/JKS)

Results indicate that scale values can be approximated satisfactorily through item-examinee sampling. Defining one observation as the response made by one examinee to one item, the similarity between the estimated scale values and normative scale values increased generally with increases in the number of observations acquired by the sampling plan.…