Automated scoring of free drawings or images as responses has yet to be used in large-scale assessments of student achievement. In this study, we propose artificial neural networks to classify these types of graphical responses from a TIMSS 2019 item. We are comparing classification accuracy of convolutional and feed-forward approaches. Our…

A two-part experiment is presented that validates a new measurement tool for scoring oral reading ability. Data collected by the U.S. government in a large-scale literacy assessment of adults were analyzed by a system called VersaReader that uses automatic speech recognition and speech processing technologies to score oral reading fluency. In the…

In this study, we explored the potential for machine scoring of short written responses to the Classroom-Video-Analysis (CVA) assessment, which is designed to measure teachers' usable mathematics teaching knowledge. We created naïve Bayes classifiers for CVA scales assessing three different topic areas and compared computer-generated scores to…

Judgmental standard setting methods have been criticized for the cognitive complexity of the judgment task that panelists are asked to complete. This study compared two methods designed to reduce this complexity: the yes/no method and the single-passage bookmark method. Two mock standard setting panel meetings were convened, one for each method,…

A nonparametric tree classification procedure is used to detect differential item functioning for items that are dichotomously scored. Classification trees are shown to be an alternative procedure to detect differential item functioning other than the use of traditional Mantel-Haenszel and logistic regression analysis. A nonparametric…

Self-deception has become a construct of great interest in individual differences research because it has been associated with levels of resilience and mental health. The Balanced Inventory of Desirable Responding (BIDR) is a self-report measure used for quantifying self-deception. In this study we used Rasch modeling to examine the properties of…

Though the Paired-Item-Score (Eakin and Long) (EJ 174 780) method of scoring true-false tests has certain advantages over the traditional scoring methods (percentage right and right minus wrong), these advantages are attained at the cost of a larger risk of misranking the examinees. (Author/BW)

Methods for setting cutoff scores based on criterion performance, normative comparison, and absolute judgment were compared for scores on a multiple-choice psychology examination for 121 undergraduates and 251 undergraduates as a comparison group. All methods fell within the standard error of measurement. Implications of differences for decision…

Whether previously noted differences in 2 sets of recommended 5-point equal-interval response anchors could have been caused by scaling too many stimuli at once was studied for scores of 110 college students. A comparison of Magnitude Estimation (MET) and Thurstone Case III illustrates the advantages of MET. (SLD)

The four scoring strategies compared were: lamda coefficients, chi-square weights, and two applications of multiple discriminant analysis. No significant differences were found when applied to the Kuder Occupational Interest Survey. (RC)

Results suggest that different groups of judges do set similar examination standards when using the same procedure, and that the average of individual judgments does not differ significantly from group consensus judgments. Significant differences were found, however, between the standards set by the two procedures employed. (RC)

Paper is concerned with the effect that ipsative scoring has upon a commonly used index of between-subtest correlation. (Author)

Two studies investigated the functional equivalence of three different scoring systems used to assess the child's ability to understand and carry out multiplicative classification tasks. All three scoring criteria produced reliable and homogeneous tests. Their factor matrices were similar, and the corresponding factor structures were invariant…