The purpose of this Item Response Theory study was to investigate how the expected reduction in item information, due to the collapsing of response categories in performance assessment data, was affected by varying testing conditions: item difficulty, item discrimination, inter-rater reliability, and direction of collapsing. The investigation used…

A method of estimating Rasch-model difficulty calibrations from judges' ratings of item difficulty is described. The ability of judges to estimate item difficulty was assessed by correlating estimated and empirical calibrations on each of four examinations offered by the American Association of State Social Work Boards. Thirteen members of the…

A Monte Carlo approach is proposed, using the Statistical Analysis System (SAS) programming language, for estimating reliability coefficients in generalizability theory studies. Test scores are generated by a probabilistic model that considers the probability for a person with a given ability score to answer an item with a given difficulty…

As performance-based assessments have gained wider use, there are increasing concerns about their dependability. This study is a synthesis of existing studies regarding the reliability or generalizability of performance assessments. The meta-analysis involves summarizing, examining, and evaluating research findings. Articles on the dependability…

The Bath County Computer Attitude Scale (BCCAS) has received limited attention concerning its reliability and validity with a U.S. adult population. As developed by G. G. Bear, H. C. Richards, and P. Lancaster in 1987, the instrument assessed attitudes toward computers in areas of computer use, computer-aided instruction, programming and technical…

Twenty-one variables believed to be important indicators of health related physical fitness were measured on male and female college students between 1991 and 1993 (n=433). Exploratory and confirmatory factor analytic techniques were used in an attempt to derive important components of physical fitness. The exploratory factor analysis identified…

This study was conducted to evaluate whether the adjectival self-description checklist may provide a viable method of quickly obtaining initial personality type information. The Personal Preferences Self-Description Questionnaire (PPDSQ) was administered to more than 420 college students, and data were analyzed using classical reliability analysis…

Interviews with children about their knowledge of a set of words was used to examine the concurrent validity of three paper-and-pencil measures of knowledge of these words--a standardized vocabulary test and two experimenter-designed tests. One experimenter-designed test, the Levels test, had three multiple-choice items per word that targeted…

A study expands on an earlier study by developing a prototypical evaluation instrument for directors of forensics. A 3-part questionnaire was constructed to assess the reliability and validity of the instrument used in the earlier study. Questionnaires were returned by 63 of 205 directors of forensics programs representing both public and private…

This digest offers principles of performance assessment as an alternative to norm-referenced tests. The definition of performance assessment developed by the U.S. Congress's Office of Technology and Assessment is given, common features are listed, and the terms "performance" and "authentic" are defined. Suggested guidelines for…

The L statistic of E. B. Page (1963) tests the agreement of a single group of judges with an a priori ordering of alternative treatments. This paper extends the two group test of D. W. Leitner and C. M. Dayton (1976), an extension of the L test, to analyze difference in consensus between two unequally sized groups of judges. Exact critical values…

The Test of Phonological Awareness (TOPA) was developed to help identify children who are delayed in their development of phonological awareness. Research supports the theory that children with poor phonological awareness are at risk of later reading difficulties. Children who score in the bottom quartile of the TOPA are considered to be at risk…

This study was conducted with three purposes in mind: (1) to determine the utility of the Colwell Music Achievement Test 3 (MAT3) as an instrument for measuring student instrumental music achievement; (2) to determine the types of musical achievement exhibited by students in Tennessee's secondary music programs; and (3) to determine the utility of…

Although videotape is the preferred method for recording behavior, there are many natural settings in which it is impractical or too intrusive. The ability to collect data in such settings and in real time has been enhanced by the advent of light, powerful laptop computers. This poster (with paper) describes a set of eight programs (developed in…

Modified Parallel Analysis (MPA) is a heuristic method for assessing "approximate unidimensionality" of item pools. It compares the second eigenvalue of the observed correlation matrix with the corresponding eigenvalue extracted from a "parallel" matrix generated by a unidimensional and locally independent model. Revised…