The item (difficulty and discrimination) and test (reliability and validity) statistics in classical test theory are highly dependent upon the calibration sample of individuals used. The estimates of item and test parameters in classical test theory is valid within a range of interest along the characteristic measured. Generally, this range of…

In an earlier pilot study of the narrative sections of Navy performance evaluations for senior enlisted personnel in pay grade E-7, it was determined by content analytic techniques that it is possible to differentiate between the performance of typical and superlative chief petty officers based on the narrative content of Evaluation Reports. A…

The Diagnostic Reading Tests were designed to assess the reading skills of college students enrolled in reading centers. To assess the reliability of the Diagnostic Reading Tests, Survey Section, Form E (DRTE), a study was conducted with university freshmen as subjects. The DRTE was administered to 31 students in an Educational Opportunity Program…

General systems for analyzing instructional interaction have found the most common teacher behavior to be asking questions. This evaluation compares and contrasts two systems for analyzing teacher questions: Price-Belland, developed by the authors from the Bloom-Saunders tradition, and Hough-Duncan, modified for detailed question analysis.…

The stepped-up reliability coefficient does not have the same standard error as an ordinary correlation coefficient. Fisher's Z -transformation should not be applied to it. Appropriate procedures are suggested. (Author)

An investigation is conducted which presents extensive Monte Carlo results which indicate the conditions under which a procedure using the F distribution can be used to study the robustness of the confidence interval procedures for small samples. A review of the literature is presented. Procedure uses a binary data matrix. Results indicate that…

A study was conducted by the Navy to develop and evaluate human performance reliability estimates for electronic maintenance. Data were collected using the Personnel Identification Information Forms, the Technical Proficiency Checkout Form, and the Job Performance Questionnaire. On the basis of the total number of uncommonly effective and the…

This study in parametric test theory deals with the statistics of reliability estimation when scores on two parts of a test follow a binormal distribution with equal (case 1) or unequal (case 2) expectations. In each case biased maximum-likelihood estimators of reliability are obtained and converted into unbiased estimators. Sampling distributions…

Given multiple independent measures of an underlying true factor and information on group membership, it is possible to compute a set of observed group means for each measure. Given at least three tests, these sets of means may be used to compute the reliability of the means for each test. The procedure for estimating true scores from the…

The use of a short, face valid, objectively scorable questionnaire to obtain students' evaluations of courses as a whole is discussed. The instrument is described from the standpoints of content domain, reliability, face validity, and university-wide applicability. This instrument would provide reliable normed data for use by campus-level…

Item options of shortened forms of the Graduate Record Examination Verbal and Quantitative tests were empirically weighted by two variants of a method originally attributed to Guttman. The first method assigned to each option of an item the mean standard score on the remaining items of all subjects choosing that option. The second procedure…

Results of a comparative study of F and Q tests, in a randomized block design with one replication per cell, are presented. In addition to these two procedures, a multivariate test was also considered. The model and test statistics, data generation and parameter selection, results, summary and conclusions are presented. Ten tables contain the…

Ten criteria for use in assessing the emphasis on environmental education in textbooks and similar resource materials were developed and given to 30 members of the Australian Conservation Foundation Education and Training Committees throughout the country. Each rater applied the criteria to three chapters of a biology textbook "The Web of…

Confidence weighting (CW) tends to improve the reliability of easy tests; the Coombs-type multiple-response (MR) option tends to improve the reliability of hard tests. It was hypothesized that, on a test of moderate difficulty, offering both the CW and MR response options would improve reliability more than either alone. Twenty-four subjects took…

Differences in design between norm referenced measures (NRM) and criterion referenced measures (CRM) are reviewed, and some of the procedures proposed on designing and evaluating CRM are examined. Differences in design of NRM and CRM are said to arise from the different purposes that underlie each measure. In addition, there are differences among…