With the iPad-mediated cognitive assessment gaining popularity with school districts and the need for alternative modes for training and instruction during this COVID-19 pandemic, school psychology training programs will need to adapt to effectively train their students to be competent in administering, scoring, an interpreting cognitive…

Through the use of excerpts from one of our own case studies, this commentary applied concepts inherent in, but not limited to, the neuropsychological literature to the interpretation of performance on the Kaufman Tests of Educational Achievement-Third Edition (KTEA-3), particularly at the level of error analysis. The approach to KTEA-3 test…

Online educational technologies offer opportunities for providing individualized feedback and detailed profiles of students' skills. Yet many technologies for mathematics education assess students based only on the correctness of either their final answers or responses to individual steps. In contrast, examining the choices students make for how…

Tatsuoka (1984) suggested several extended caution indices and their standardized versions that have been used as person-fit statistics by researchers such as Drasgow, Levine, and McLaughlin (1987), Glas and Meijer (2003), and Molenaar and Hoijtink (1990). However, these indices are only defined for tests with dichotomous items. This paper extends…

This study utilizes an argument-based approach to validation to examine the implications of reliability in order to further differentiate the concepts of score and decision consistency. In a methodological example, the framework of generalizability theory was used to estimate appropriate indices of score consistency and evaluations of the…

In this study, the statistical properties of extended caution indices are investigated, and their relationships to Guttman scales and to item and person response curves are discussed. Further, these indices are standardized, and an example of their potential usefulness for diagnosing students' misconceptions is shown. (Author/BW)

Analyzed 56 Wechsler Intelligence Scale for Children-Revised protocols completed by 1 certified and 8 licensed practitioners to examine administration and scoring mistakes. Observed numerous mistakes (failure to record examinee responses, assigning too few or too many points to answers, inappropriate questioning, and failure to obtain correct…

Examined performance and error patterns of 50 bilingual Mexican-American third through fifth graders on the Expressive One-Word Picture Vocabulary Test (EOWPVT) and the Peabody Picture Vocabulary Test-Revised (PPVT-R). The EOWPVT had adequate concurrent validity with PPVT-R. On both tests, scores were almost two standard deviations below normative…

Two indices for measuring the degree of conformity or consistency of an individual examinee's response pattern on a set of items are developed. The use of the indices for spotting aberrant response patterns of examinees is detailed. (Author/JKS)

The Sato Caution Index takes into account the number and difficulty of items gotten wrong by a student within his or her ability, as well as the number and difficulty of items gotten right beyond his or her ability. Sato subtracts the two components to define a single Caution Index. In this study, the components are kept separate, defining a…

Several extended caution indices (ECIs) have been introduced earlier as a link between two distinctly different approaches: one based on standard statistics and the other, a model-based approach, utilizing item response theory (IRT). Expected values and variance of some ECIs are derived and their statistical properties are compared and discussed.…

This study introduces the individual consistency index (ICI), which measures the extent to which patterns of responses to parallel sets of items remain consistent over time. ICI is used as an error diagnostic tool to detect aberrant response patterns resulting from the consistent application of erroneous rules of operation. (Author/PN)

The Guttman scale is discussed from the viewpoint of errors in response patterns. The errors are assumed to be distributed as a binomial. A double-barreled significance test is suggested having two probabilities: high probability and low probability. (Author)

College Level Examination Program (CLEP) Tests were normed on a national basis, administering the test to nationwide samples of subjects. Norms appear in the booklet, CLEP Scores: Interpretation and Use, and consist of the test score means for groups of students receiving grades of A, B, C, D, and F in the relevant course, the proportion of…

The advantages of a rule assessment approach to interpreting achievement test results were demonstrated using an S-P chart with coded error types. The problems of similar total test scores resulting from completely different misapprehensions, as well as correct answers resulting from incorrect rules of operation, were addressed using simulated…