This study investigated cognitive constructs to be measured by word problems in algebra. One performance-based assessment was administered to 290 high school students. Students' responses were scored by three scoring systems: the correct/incorrect criterion (0/1); a holistic scoring rubric (0-4); and an analytical scoring rubric for measuring…

Ten factors relevant to the standard-setting decision on important tests (i.e., high school diploma tests) are proposed as a basis for a standard-setting preparation program. It is suggested that, typically, those who set standards have had little or no experience and would benefit from formal preparation. (SLD)

Issues concerning the selection of judges for standard setting are discussed. Determining the consistency of judges' recommendations, or their congruity with other expert recommendations, would help in selection. Enough judges must be chosen to allow estimation of recommendations by an entire population of judges. (SLD)

Training judges to generate item ratings in standard setting once the reference group has been defined is discussed. It is proposed that sensitivity to the factors that determine difficulty can be improved through training. Three criteria for determining when training is sufficient are offered. (SLD)

The continuous method of scoring a performance test composed of standardized patients was compared with a derivative method that assigned each of the 131 examinees (medical residents) a dichotomous score, and use of Angoff's method with these scoring methods was studied. Both methods produce reasonable means and distributions of scores. (SLD)

An approximation to the likelihood for the generalized linear models with random coefficients is derived and is the basis for an approximate Fisher scoring algorithm. The method is illustrated on the logistic regression model for one-way classification, but it has an extension to the class of generalized linear models and to more complex data…

The item response function (IRF) for a polytomously scored item is defined as a weighted sum of the item category response functions (ICRF, the probability of getting a particular score for a randomly sampled examinee of ability theta). This paper establishes the correspondence between an IRF and a unique set of ICRFs for two of the most commonly…

The types of test score interpretive information considered useful to failing examinees were studied through interviews with Educational Testing Service (ETS) staff members. Research literature on interpretive information for failing test takers was reviewed, and procedures currently used at ETS were determined. Managers of 23 testing programs…

Research supported by the National Research Council of Canada, Grant APA-252-2057-B.

A critical review of systems of scoring multiple choice tests is presented and the superiority of a system based upon elimination method over one based upon the best answer mode is hypothesized. This is discussed in terms of the capacity of the mode to reveal the relationships among decoy options and the effects of partial information,…

This is a psychometric hoax paper, the purpose of which is to indicate once again the importance of cross-validation, particularly in the development of specially-keyed inventories. The junior author and the new psychometric method play critical roles in the study. Appropriate credit and references are present. (Author)

This manual provides instructions for scoring the Teacher Concerns Statement, a questionnaire for pre- and inservice teachers consisting of one open-ended question asking for the respondent's concerns when he/she thinks about teaching. The Questionnaire, a copy of which is provided, consists of five questions asking for back ground information…

Smith (1969) reported the results of an instrument for measuring teacher judgment of written composition. His test was first administered to a group of "experts" whose ratings were in high agreement. Then the test was given to a sample of over 200 teachers and lay readers. Among Smith's conclusions was that over half of the teachers have judgment…