The authors propose a concise formula to evaluate the standard error of the estimated latent variable score when the true values of the structural parameters are not known and must be estimated. The formula can be applied to factor scores in factor analysis or ability parameters in item response theory, without bootstrap or Markov chain Monte…

A new measure for reliability of a rating scale is introduced, based on the classical definition of reliability, as the ratio of the true score variance and the total variance. Clinical trial data can be employed to estimate the reliability of the scale in use, whenever repeated measurements are taken. The reliability is estimated from the…

Presents and illustrates an appropriate formula for correction for attenuation that can be used in situations in which one measure includes another measure as its part. The formula can be used for computing the correlation coefficient for true scores between total test and part test. (SLD)

Dorans and Holland (2000) and von Davier, Holland, and Thayer (2003) introduced measures of the degree to which an observed-score equating function is sensitive to the population on which it is computed. This article extends the findings of Dorans and Holland and of von Davier et al. to item response theory (IRT) true-score equating methods that…

Physician-patient communication is a clinical skill that can be learned and has a positive impact on patient satisfaction and health outcomes. A concerted effort at all medical schools is now directed at teaching and evaluating this core skill. Student communication skills are often assessed by an Objective Structure Clinical Examination (OSCE).…

A hands-on activity is described in which students attempt to measure something that they cannot see. In small groups, students estimate the number of marbles in sealed boxes. Next, students' estimates are compared with the actual numbers. Last, values from both the students' estimates and actual numbers are used to explain measurement theory and…

A Monte Carlo simulation technique for generating dichotomous item scores is presented that implements (a) a psychometric model with different explicit assumptions than traditional parametric item response theory (IRT) models, and (b) item characteristic curves without restrictive assumptions concerning mathematical form. The four-parameter beta…

A Markov chain Monte Carlo (MCMC) method and a bootstrap method were compared in the estimation of standard errors of item response theory (IRT) true score equating. Three test form relationships were examined: parallel, tau-equivalent, and congeneric. Data were simulated based on Reading Comprehension and Vocabulary tests of the Iowa Tests of…

Makes the case that the arguments of F. Schmidt and J. Hunter in favor of the correction for attenuation in theory testing are based on mistaken assumptions. Outlines arguments against the routine use of correction for attenuation, focusing on the relationship between true scores and construct scores. (SLD)

Latent variable models for an infinite sequence (or universe) of manifest variables that may be discrete, continuous, or a combination of both, are considered. A main theorem is presented that characterizes when it is possible to construct latent variable models that satisfy unidimensionality, monotonicity, conditional independence, and tail…

Describes an idiosyncracy of the MULTILOG (D. Thissen, 1991) parameter estimation process discovered during a simulation study involving the graded response model. A misordering reflected in boundary function location parameter estimates resulted in a large negative contribution to the true score followed by a large positive contribution. These…

This study aimed at estimating the correlation between true scores (true consistency) of vocational interest over a short time span in a sample of 1089 adults. Participants were administered 54 items assessing vocational, family, and leisure interests twice over a 1-month period. Responses were analyzed with a multitrait (MT) model, which supposes…

In educational testing, subscores may be provided based on a portion of the items from a larger test. One consideration in evaluation of such subscores is their ability to predict a criterion score. Two limitations on prediction exist. The first, which is well known, is that the coefficient of determination for linear prediction of the criterion…

The idea of test consistency is illustrated, with reference to two sets of test scores. A mathematical model is used to explain the relative consistency and relative inconsistency of measurements, and a means of indexing reliability is derived using the model. Practical aspects of estimating reliability are considered. (TJH)