The authors describe the initial development of the Wagner Assessment Test (WAT), an instrument designed to assess critical thinking, using the 5-faceted view popularized by the Watson-Glaser Critical Thinking Appraisal (WGCTA; G. B. Watson & E. M. Glaser, 1980). The WAT was designed to reduce the degree of successful guessing relative to the…

Should instruments, such as Likert-type scales, contain both positively worded and negatively worded items within the same scale (i.e. mixed format)? Recent evidence suggests that the use of scales with a mixed format can adversely affect the psychometric properties of scales. In particular, the mean item response to the positively worded items…

Rasch analysis was used to illustrate the usefulness of item-level analyses for evaluating a common therapy outcome measure of general clinical distress, the Symptom Checklist-90-Revised (SCL-90-R; Derogatis, 1994). Using complementary therapy research samples, the instrument's 5-point rating scale was found to exceed clients' ability to make…

This article describes the development of a screening instrument for young children. Screening items were tested first in a non-selected population of children aged 8-20 months (n = 478). Then, parents of children with clinically diagnosed ASD (n = 153, average age 87 months) or ADHD (n = 76, average age 112 months) were asked to score the items…

Many districts and schools across the U. S. have begun to develop and administer assessments to complement state testing systems and provide additional information to monitor curriculum, instruction and schools. In advance of this trend, the Jackson Public Schools (JPS) district has had a district benchmark testing system in place for many years.…

In a course on biological psychology and neuropsychology, study questions were provided that also appeared as test questions in the course exam. This method was introduced to support students in active processing and reproduction of the study texts, and study planning. Data were gathered to test the hypothesis that study question use would be…

When standardized English-language tests are administered to test takers worldwide, the test-taking population could be varied on a number of personal and educational characteristics such as age, gender, first language, and academic discipline. As test tasks and test items may not always be prepared keeping this diversity of characteristics in…

This study compared four methods for setting item response time thresholds to differentiate rapid-guessing behavior from solution behavior. Thresholds were either (a) common for all test items, (b) based on item surface features such as the amount of reading required, (c) based on visually inspecting response time frequency distributions, or (d)…

This study investigated the relationship between readability of 73 text-only multiple-choice questions from Trends in International Mathematics and Science Study (TIMSS) 2003 and performance of two groups of South African learners: those with limited English-language proficiency (learners attending African schools) and those with better…

Computer-based assessments are becoming more commonplace, perhaps as a necessity for faculty to cope with large class sizes. These tests often occur in large computer testing venues in which test security may be compromised. In an attempt to limit the likelihood of cheating in such venues, randomised presentation of items is automatically…

A study was conducted to begin a process of validating hypothesized causes of local item dependence (LID) in large-scale performance assessments. Data for the study are item level scores from 26 science tasks from the 1993 edition of the Maryland School Performance Assessment Program. Causes of high LID were hypothesized from studies by Ferrara et…

Response generative modeling (RGM) is an approach to psychological measurement that involves a "grammar" capable of assigning a psychometric description to every item in a universe of items and is capable of generating all the items in that universe. The article discusses the rationale behind RMG and its roots, explores how it relates to…

This study examines the predictability of Graduate Record Examinations (GRE) reading item difficulty (equated delta) for the three major reading item types: main idea, inference, and explicit statement items. Each item type is analyzed separately, using 110 GRE reading passages and their associated 244 reading items; selective analyses of 285…

Prediction of the difficulty (equated delta) of a large sample (n=213) of reading comprehension items from the Test of English as a Foreign Language (TOEFL) was studied using main idea, inference, and supporting statement items. A related purpose was to examine whether text and text-related variables play a significant role in predicting item…

Issues concerning the cost and feasibility of using Formulating Hypotheses (FH) test item types for the Graduate Record Examinations have slowed research into their use. This project focused on two major issues that need to be addressed in considering FH items for operational use: the costs of scoring and the assignment of scores along a range of…