One of the challenges of discussing validity arguments for machine scoring of essays centers on the absence of a commonly held definition and theory of good writing. At best, the algorithms attempt to measure select attributes of writing and calibrate them against human ratings with the goal of accurate prediction of scores for new essays.…

As with any psychometric models, the validity of inferences from cognitive diagnosis models (CDMs) determines the extent to which these models can be useful. For inferences from CDMs to be valid, it is crucial that the fit of the model to the data is ascertained. Based on a simulation study, this study investigated the sensitivity of various fit…

Research has often found that, when high school grades and SAT scores are used to predict first-year college grade-point average (FGPA) via regression analysis, African-American and Latino students, are, on average, predicted to earn higher FGPAs than they actually do. Under various plausible models, this phenomenon can be explained in terms of…

Two conventional scores and a weighted score on a group test of general intelligence were compared for reliability and predictive validity. (Editor)

Data from a postdictive study of the tests of the Graduate Record Examination and the eight semesters of undergraduate grade averages, each semester's average being computed independently of the rest, are presented. (Editor)

Despite the numerous individual illustrations in the literature showing how the discrimination index may be used to identify items with faults, its overall effectiveness as a measure of item quality, defined in terms of the presence or absence of faults, is not clear. This study investigates its validity. (Author/RK)

Alternative scoring methods yield essentially the same information, including scale intercorrelations and validity. Reasons for preferring the traditional psychometric scoring technique are offered. (Author/AG)

When the precise basis of selection effect on correlation and regression equations is unknown but can be modeled by selection on a variable that is highly but not perfectly related to observed scores, the selection effects can lead to the commonly observed "overprediction" results in studies of predictive bias. (Author/PN)

This review of issues about correcting for attenuation concludes that the basic difficulty lies in being able to identify and equate sources of variance in estimates of validity and reliability. Recommendations are proposed for cautious use of correction for attenuation. (Author/CM)

The purpose of this study was to gather empirical evidence to compare the reliabilities and concurrent validities of multiple choice and true-false tests that were written to measure understandings and relationships in the same content areas. (Author)

Meta-analysis of research on a topic may exclude reports that do not present their results in a statistical form amendable to summarizing procedures used. This article illustrates how researchers can consider this aspect in computing statistics from their data. An illustration of how that can sometimes be done is presented. (Author/JAZ)

The Self-Description Questionnaire III (SDQ III) was designed to measure 13 factors of self-concept. The reliabilities of the factors were high, and correlations among them were low. Academic achievement measures in language and mathematics were correlated with self-concepts in the same areas but not with other self-concept factors. (Author/DWH)

The present study suggests that although the reliability of an academic aptitude test given under formula-score condition can be increased substantially through empirical option weighting, much of the increase is due to the capitalization of the keying procedure on omitting tendencies which are reliable but not valid. (Author)

The purpose of this paper was to examine the problems and opportunities of academic prediction for different ethnic groups. (Editor)

Over 2,000 writing samples were collected from four undergraduate institutions and compared, where possible, with scores on a multiple-choice test. High correlations between ratings of the writing samples and multiple-choice test scores were obtained. Samples contributed substantially to the prediction of both college grades and writing…