Results of a reanalysis of previously published data (B. M. Byrne, 1989) support the correlated uniqueness model, diagnostic tests of the validity of confirmatory factor analysis (CFA), multitrait multimethod (MTMM) solutions, inclusion of external validity in MTMM design, and application of factorial invariance to test stability of CFA-MTMM…

Three analytic approaches were used in a framework of classical test theory to examine the construct validity of a mathematics assessment of 16 constructed response items. Results from 280 elementary school students across four age groups suggest a developmental structure of tasks and subdomains that was generally consistent with the test's…

This volume is the fourth in a series for college faculty and advanced graduate students, "Survival Skills for Scholars." It offers practical advice for developing, using, and grading classroom examinations, focusing on traditional multiple-choice and constructed-response tests rather than alternative assessments. (SLD)

Coefficient alpha, the most commonly used estimate of internal consistency, is often considered a lower bound estimate of reliability, though the extent of its underestimation is not typically known. Many researchers are unaware that coefficient alpha is based on the essentially tau-equivalent measurement model. It is the violation of the…

Multiple-choice testing (MCT) has several advantages which are becoming more relevant in the current financial climate. In particular, they can be machine marked. As an objective testing method it is particularly relevant to engineering and other factual courses, but MCTs are not widely used in engineering because students can benefit from…

This paper compares the approach and resultant outcomes of item response models (IRMs) and classical test theory (CTT). First, it reviews basic ideas of CTT, and compares them to the ideas about using IRMs introduced in an earlier paper. It then applies a comparison scheme based on the AERA/APA/NCME "Standards for Educational and…

In May 2001, students in the author's Advanced Placement (AP) United States History class were embroiled in a controversy surrounding the AP exam, in particular, having access to the exam's Document Based Question (DBQ) and free response portion prior to the test's administration. Prior to the exam, the College Board had provided a fifty-year time…

The construct validity of the Australian version of the Multidimensional School Anger Inventory-Revised (MSAI-R) was examined using exploratory factor analysis (EFA), Rasch analysis, and confirmatory factor analysis (CFA) on a sample of 1,400 Australian students enrolled in Years 8 through 12. The EFA revealed a strong replication of the MSAI-R's…

This research paper describes the benefits of using an activity-based rhetorical perspective to develop English for specific purposes (ESP) test specifications. This approach expands the potential of ESP test specifications to analyze and describe target language use (TLU) situations, TLU tasks, and ESP test tasks. Multiple activity systems are…

The web-based formative assessment developed in this research is named Formative Assessment Module of the Web-based Assessment and Test Analysis System (FAM-WATA). FAM-WATA is a multiple-choice web-based formative assessment module containing six effective strategies: 'repeat the test', 'correct answers are not given', 'query scores', 'ask…

Brief explanations are presented of some of the different methods used to score multiple-choice tests; and some studies of partial information, guessing strategies, and test-taking behaviors are reviewed. Studies are grouped in three categories of effort to improve scoring: (1) those that require extra effort from the examinee to answer…

This handbook addresses the complex and expanding vocabulary of performance assessment and measurement by providing a glossary of related terms and lists of resources for the student, practitioner, and policymaker. Performance assessment includes all forms of such assessment from multiple choice and paper-and-pencil tests to alternative…

A general issue related to language program development involves the empirical rationalization of cut score decisions in criterion-referenced language tests. Cut score dependability focuses on the consistency of the decisions in repeated testing or the assessment of language learner performances. In this case, the issue is to determine the optimal…

This study of the validity of the Graduate Record Examinations (GRE) General Test used data from predictive validity studies that were conducted by the GRE Validity Study Service (VSS) in 79 graduate departments. The performance criterion was first-year grades in graduate school. Observed validities were computed, and for each graduate department…

Monte Carlo methods were employed to assess the relative power of the paired samples t test and Wilcoxon's signed-ranks test under ten population shapes. Results of the study indicated that: (1) each of the two statistics was more powerful than the other in given situations; (2) the power advantages of the t test under normal theory were small;…