Power analysis is a fundamental prerequisite for conducting scientific research. Without power analysis the researcher has no way of knowing whether the sample size is large enough to detect the effect he or she is looking for. This paper demonstrates how psychometric factors such as measurement error and equating error affect the power of…

A recent comparative analysis of alternative interval estimation approaches and procedures has shown that confidence intervals (CIs) for true raw scores determined with the Score method--which uses the normal approximation to the binomial distribution--have actual coverage probabilities that are closest to their nominal level. It has also recently…

Background: A recent article published in "Educational Research" on the reliability of results in National Curriculum testing in England (Newton, "The reliability of results from national curriculum testing in England," "Educational Research" 51, no. 2: 181-212, 2009) suggested that: (1) classification accuracy can be…

Examines the question of how large a sample must be in order to produce empirical Bayes estimates which are preferable to other commonly used estimates, such as proportion correct observed score. (Author/RC)

Classical test theory findings can be derived from the concepts of conditional expectation, conditional independence, and related notions. It is shown that these concepts provide precisely the formalism needed to obtain the classical results with minimal assumptions and with greatest economy in the methods of proof. (RC)

From a decision theoretic viewpoint, a general coefficient (delta) for tests is derived. The coefficient is applied to three kinds of decision situations. The first situation involves a true score estimated by a function of the observed score of a subject on a test (point estimation). Using the squared error loss function and Kelley's formula for…

To resolve a recent controversy between Klein and Cleary and Levy, a model for dichotomous congeneric items is presented which has mean errors of zero, dichotomous true scores that are uncorrelated with errors, and errors that are mutually uncorrelated. (Author)

Ordinal measurement is the rank ordering of individuals in a population. For ordinal measurement, the concept of an individual propensity distribution is his or her true score. Estimation of, as well as other aspects of the distribution, are discussed. (Author/JKS)

Classical test theory is based on the concept of a true score for each examinee, defined as the expected or average score across an infinite number of repeated parallel tests. In most cases, there is only a score from a single administration of the test in question. The difference between this single observed score and the underlying true score is…

The problem of whether a precise connection exists between the stochastic processes considered in mathematical learning theory and the Guttman simplex is investigated. The approach used is to derive a set of conditions which a probabilistic model must satisfy in order to generate inter-trial correlations with the perfect simplex property.…

The relationship between true ability (operationally defined as the number of items for which the examinee actually knew the correct answer) and the effects of guessing upon observed test variance was investigated. Three basic hypotheses were treated mathematically: there is no functional relationship between true ability and guessing success;…

Assuming errors of measurement are distributed binomially, this article reviews various procedures for constructing an interval for an individual's true number-correct score; presents two general interval estimation procedures for an individual's true scale score (i.e., normal approximation and endpoints conversion methods); compares various…

A modified beta binomial model is presented for use in analyzing random guessing multiple choice tests and taste tests. Detection probabilities for each item are distributed beta across the population subjects. Properties for the observable distribution of correct responses are derived. Two concepts of true score estimates are presented.…

Using the concepts of conditional probability, conditional expectation, and conditional independence, the main results of the classical test theory model can be derived in a very few steps with minimal assumptions. The present effort explores the possibility that present classical test theories can be further condensed. (Author/RC)