The literature on Automated Essay Scoring (AES) systems has provided useful validation frameworks for any assessment that includes AES scoring. Furthermore, evidence for the scoring fidelity of AES systems is accumulating. Yet questions remain when appraising the scoring performance of AES systems. These questions include: (a) which essays are…

Divergent thinking (DT) tests are very often used in creativity studies. Certainly DT does not guarantee actual creative achievement, but tests of DT are reliable and reasonably valid predictors of certain performance criteria. The validity of DT is described as reasonable because validity is not an all-or-nothing attribute, but is, instead, a…

Adults took the Rod and Frame, Portable Rod and Frame, and Embedded Figures Tests. Absolute and algebraic frame-effect scores were more reliable and valid than rod-effect algebraic scores. Correlations with the Embedded Figures Test were so low that the interchangeability of these field articulation measures is questionable. (Author/RD)

On distractor-identification tests students mark as many distractors as possible on each test item. A grading scale is developed for this type testing. The score is optimal in that it yields an unbiased estimate of the student's score as if no guessing had occurred. (Author/JKS)

Six different approaches to scoring test data, including number right, correction for guessing, and answer-until-correct, were investigated using Monte Carlo techniques. Modes permitting multiple response showed higher internal consistency, but there was little difference among modes for a validity measure. (JKS)

Paper is concerned with the effect that ipsative scoring has upon a commonly used index of between-subtest correlation. (Author)

Procedures for computing content validity and consistency reliability coefficients and determining the statistical significance of these coefficients are described. Procedures employing the multinomial probability distribution for small samples and normal curve probability estimates for large samples, can be used where judgments are made on…

In this paper, an attempt has been made to synthesize some of the current thinking in the area of criterion-referenced testing as well as to provide the beginning of an integration of theory and method for such testing. Since criterion-referenced testing is viewed from a decision-theoretic point of view, approaches to reliability and validity…

A potentially valuable measure of overconfidence on probabilistic multiple-choice tests was evaluated. The measure of overconfidence was based on probabilistic responses to nonsense items embedded in a vocabulary test. The test was administered under both confidence response and conventional choice response directions to 208 undergraduate…

A procedure for scoring multiple-choice tests by assigning different weights to every option of a test item is investigated. The weighting method used was based on that proposed by Davis, which involves taking the upper and lower 27% of a sample, according to some criterion measure, and using the percentages of these groups marking an item option…

The benefits of using essay tests rather than objective tests in professional education programs are discussed. Essay tests offer practice in writing, creativity and formal communications. Guidelines for using and scoring a sample essay test in biology are presented. (BH)

Estimates for the variance of empirically determined scoring weights are given. It is shown that test item writers should write distractors that discriminate on the criterion variable when this type of scoring is used. (Author)

This study compares various item option scoring methods with respect to coefficient alpha and a concurrent validity coefficient. The scoring methods under consideration were: (1) formula scoring, (2) a priori scoring, (3) empirical scoring with an internal criterion, and (4) two modifications of formula scoring. The study indicates a clear…

A student's choice of an answer to a test question is a coarse measure of his knowledge about the subject matter of the question. Much finer measurement might be achieved if the student were asked to estimate, for each possible answer, the probability that it is the correct one. Such a procedure could yield two classes of benefits: (a) students…

The objective of this study was to show that standardized reading scores could be adequately estimated from scores on a criterion-referenced test in reading. This would reduce classroom test time, while, at the same time, provide the kinds of information teachers need to guide instruction, and the kinds of information administrators require for…