Experimental psychology is witnessing an increase in research on individual differences, which requires the development of new tasks that can reliably assess variations among participants. To do this, cognitive researchers need statistical methods that many researchers have not learned during their training. The lack of expertise can pose…

Successful testing programs rely on high-quality test items to produce reliable scores and defensible exams. However, determining what statistical screening criteria are most appropriate to support these goals can be daunting. This study describes and demonstrates cost-benefit analysis as an empirical approach to determining appropriate screening…

Introduction: The aim of this study was to analyze the relationship between sociometric types, behavioral categories and self-attributions for academic failure ("Ability", "Effort" or "External Causes") in "Reading", "Mathematics" and "General". Method: The total sample was composed of…

Multiple matrix designs are commonly used in large-scale assessments to distribute test items to students. These designs comprise several booklets, each containing a subset of the complete item pool. Besides reducing the test burden of individual students, using various booklets allows aligning the difficulty of the presented items to the assumed…

A reverse engineering approach to automatic item generation (AIG) was applied to a figure-based publicly released test item from the Organisation for Economic Cooperation and Development (OECD) Programme for International Student Assessment (PISA) mathematical literacy cognitive instrument as part of a proof of concept. The author created an item…

Reliability is a property of test scores from individuals who have been sampled from a well-defined population. Reliability indices, such as coefficient and related formulas for internal consistency reliability (KR-20, Hoyt's reliability), yield lower bound reliability estimates when (a) subjects have been sampled from a single population and when…

When performing large scale evaluations (e.g., on a state-wide or national level) it may not be possible to administer all items in the item universe to all respondents in the subject population. One method which has been proposed to sample both items and respondents is multiple matrix sampling (MMS) in which a sample of the items is administered…

Shoemaker's conclusions related to the influence of various data base characteristics (reliability, variability of item difficulty indices, and degree of skewness in the normative distribution) on the standard error of a mean estimated via multiple matrix sampling procedures are examined. (Author/RC)

Results indicate that scale values can be approximated satisfactorily through item-examinee sampling. Defining one observation as the response made by one examinee to one item, the similarity between the estimated scale values and normative scale values increased generally with increases in the number of observations acquired by the sampling plan.…

The distributional characteristics of Kaiser's Measure of Sampling Adequacy (MSA) were investigated in sample matrices generated from multivariate normal populations of specified correlation levels. Systematic variation of sample size and number of variables revealed the overall MSA to be most influenced by the number of variables. (Author/JKS)

This investigation had two major purposes. The first was to explore the use of an inferential technique called Tukey's Jackknife in establishing a confidence interval about cooefficient alpha reliability. The second purpose was to study the robustness of the Feldt and the jackknife procedures when the data fails to satisfy usual normality…

Described herein are formulas and computational procedures for estimating the mean and second through fourth central moments of universe scores through multiple matrix sampling. Additionally, procedures are given for approximating the standard error associated with each estimate. All procedures are applicable when items are scored either…

Standard errors of pooled mean estimate in multiple matrix sampling were compared for two procedures. The data were from tests involving items with and without replacement. The two procedures involve the formulations of Madow and Lord, and Novick; the former permits sampling of item, with or without replacement, whereas the latter is to be used…

A model for multiple choice test-taking behavior is proposed which is different from those presently used for item sampling theory. A new theory is developed, which includes a new concept to facilitate comprehension of item sampling theory, a "number-known" score distribution. A major advantage of the model is that it accommodates data from…