The bookmark standard-setting procedure is an item response theory-based method that is widely implemented in state testing programs. This study estimates standard errors for cut scores resulting from bookmark standard settings under a generalizability theory model and investigates the effects of different universes of generalization and error…

Studied three procedures for estimating the standard errors of school passing rates using a generalizability theory model and considered the effects of student sample size. Results show that procedures differ in terms of assumptions about the populations from which students were sampled, and student sample size was found to have a large effect on…

Investigated incorporating a testlet definition into the estimation of the conditional standard error of measurement (SEM) for tests composed of testlets using five conditional SEM estimation methods. Results from 3,876 tests from the Iowa Tests of Basic Skills and 1,000 simulated responses show that item-based methods provide lower conditional…

Previous studies have indicated that the reliability of test scores composed of testlets is overestimated by conventional item-based reliability estimation methods (S. Sireci, D. Thissen, and H. Wainer, 1991; H. Wainer, 1995; H. Wainer and D. Thissen, 1996; G. Lee and D. Frisbie). In light of these studies, it seems reasonable to ask whether the…

The primary purpose of this study was to investigate the appropriateness and implication of incorporating a testlet definition into the estimation of the conditional standard error of measurement (SEM) for tests composed of testlets. The five conditional SEM estimation methods used in this study were classified into two categories: item-based and…

Studied the effects of items, passages, contents, themes, and types of passages on the reliability and standard errors of measurement for complex reading comprehension tests using seven different generalizability theory models. Results suggest that passages and themes should be taken into account when evaluating the reliability of test scores for…

Studied the appropriateness and implications of incorporating a testlet definition into the estimation of procedures of the conditional standard error of measurement (SEM) for tests composed of testlets. Simulation results for several methods show that an item-based method using a generalizability theory model provided good estimates of the…

The purpose of this study was to investigate the relative appropriateness of several procedures for estimating reliability and standard errors of measurement of complex reading comprehension tests. Seven generalizability theory models were conceptualized by incorporating one or several factors of items, passages, themes, contents, and types of…

The percentage of students at/above a cut point (PAAC) is one of the most common measures used for reporting school-level performance relative to a proficiency standard (L. Cronbach, N. Bradburn, and D. Horvitz, 1994). The two purposes of this study were to introduce procedures for estimating standard errors for school PAACs under a…

The Bookmark Standard Setting Procedure (Lewis, Mitzel, and Green, 1996) is an item-response-theory-based standard setting method that has been widely implemented by state testing programs. The primary purposes of this study were to: (1) estimate standard errors for cutscores that result from Bookmark standard settings under a generalizability…

School test performance is commonly summarized in terms of the percentage of students at or above a cut score (PAAC) that has been set on a test. Two approaches to estimating the standard errors for school PAACs were examined in this study: conditional standard errors and overall standard errors. The tests used were English language arts and…