Although content validity--how well each item of an instrument represents the construct being measured--is foundational in the development of an instrument, statistical validity is also important to the decisions that are made based on the instrument. The primary purpose of this study is to demonstrate how simulation studies can be used to assist…

Comparing variability of test scores across alternate forms, test conditions, or subpopulations is a fundamental problem in psychometrics. A confidence interval for a ratio of standard deviations is proposed that performs as well as the classic method with normal distributions and performs dramatically better with nonnormal distributions. A simple…

Test scores that are not perfectly reliable cannot be strictly equated unless they are strictly parallel. This fact implies that tau equivalence can be lost if an equipercentile equating is applied to observed scores that are not strictly parallel. Thirty-six simulated data sets are produced to simulate equating tests with different difficulties…

An Educational Testing Service (ETS) procedure was evaluated, which is based on item response theory and estimates true scores on tests not taken. The reading, vocabulary, and mathematics tests of high school seniors from the National Longitudinal Study (NLS) of 1972 and the High School and Beyond (HSB) seniors of 1980 and 1982 were found to share…

The concept of testing for partial knowledge is considered with the concept of tailored testing. Following the special usage of latent trait theory, the word valdity is used to mean the correlation of a test with the construct the test measures. The concept of a method factor in the test is also considered as a part of the validity. The possible…

One hundred twenty-eight observers randomly assigned to 16 treatment conditions in a modified Latin square design, viewed three videotapes of simulated classrooms in which teacher behavior was controlled (paralleling psychophysical procedures) to fit unambiguously into specific categories on ratings of frequency and variety of social…

A computer simulation procedure was developed to reproduce the overall pattern of results obtained in the Educational Testing Service Growth Study. Then simulated data for seven sets of 10,000 to 15,000 cases were analyzed, and findings compared on the basis of correlations between estimated and true growth scores. Findings showed that growth was…

The Test of English as a Foreign Language (TOEFL) was used in this study, which attempted to develop a new methodology for assessing the speededness of right-scored tests. Traditional procedures of assessing speededness have assumed that the test is scored under formula-scoring instructions; this approach is not always appropriate. In this study,…

A set of tests that might be reasonably used as provisional criterion measures in research on scientific thinking, particularly creative thinking, were developed and an assessment was made of the suitability of these tests as criterion variables from the standpoint of their psychometric properties. The Tests of Scientific Thinking are performance…

This paper describes the procedures and results of a pilot study of a computer-based System of Interactive Guidance and Information (SIGI) which was designed to help community college students make rational career decisions. A number of the important features of the system are summarized in terms of the source of the value dimensions used in SIGI,…

Development of The Assessment of Basic Competencies (ABC), a test battery based on the three-parameter logistic model, is described. Eleven dimensions of intellectual growth are measured, from the pre-kindergarten level through ninth grade. An educationally relevant skill domain is represented by each test. Unique properties of the test, based on…