This study examined the relative effectiveness of the multidimensional bi-factor model and multidimensional testlet response theory (TRT) model in accommodating local dependence in testlet-based reading assessment with both dichotomously and polytomously scored items. The data used were 14,089 test-takers' item-level responses to the testlet-based…

There are persistent challenges to teaching mixed methods and innovative solutions are sought in order to address the needs of an increasingly diverse global audience seeking mixed methods instruction. This mixed methods study was conducted to gain insights to course design by more fully understanding the relationships among graduate student…

The present study reports on the construction of a research instrument developed to examine preschool teachers' beliefs and practices in relation to emergent literacy. A 130-item survey (Preschool Literacy Survey, PLS) was completed by a total of 90 preschool teachers in Norway. Items were grouped into homogenous scales, and the relationship…

Chemistry departments have felt pressure in recent years to produce quality data on student achievement of learning outcomes. External (e.g., accreditation agencies) and internal (e.g., academic deans) entities are demanding regular review of student achievement. It is thus necessary for the chemistry community to develop valid and reliable…

Common test items play an important role in equating alternate test forms under the common item nonequivalent groups design. When the item response theory (IRT) method is applied in equating, inconsistent item parameter estimates among common items can lead to large bias in equated scores. It is prudent to evaluate inconsistency in parameter…

Cognitive diagnostic computerized adaptive testing (CD-CAT) purports to combine the strengths of both CAT and cognitive diagnosis. Cognitive diagnosis models aim at classifying examinees into the correct mastery profile group so as to pinpoint the strengths and weakness of each examinee whereas CAT algorithms choose items to determine those…

Although calibration has been widely studied, questions remain about how best to capture confidence ratings, how to calculate continuous variable calibration indices, and on what exactly students base their reported confidence ratings. Undergraduates in a research methods class completed a prior knowledge assessment, two sets of readings and…

In this study, we investigated differential item functioning (DIF) and its sources using a latent class (LC) modeling approach. Potential sources of LC DIF related to instruction and teacher-related variables were investigated using substantive and three statistical approaches: descriptive discriminant function, multinomial logistic regression,…

A primary goal of standards-based statewide achievement tests is to classify students into achievement levels that enable valid inferences about student content area knowledge and skill. Explicating how knowledge and skills are expected to differ in complexity in achievement level descriptors, and how that complexity is related to empirical item…

Evidence-centered design (ECD) is an orientation towards assessment development. It differs from conventional practice in several ways and consists of multiple activities. Each of these activities results in a set of useful documentation: domain analysis, domain modeling, construction of the assessment framework, and assessment…

Item purification is an iterative process that is often advocated as improving the identification of items affected by differential item functioning (DIF). With test-score-based DIF detection methods, item purification iteratively removes the items currently flagged as DIF from the test scores to get purified sets of items, unaffected by DIF. The…

In an item-method-directed forgetting task, Chinese words were presented individually, each followed by an instruction to remember or forget. Colored probe items were presented following each memory instruction requiring a speeded color-naming response. Half of the probe items were novel and unrelated to the preceding study item, whereas the…

This study examines the English section of the university entrance examination in Myanmar in terms of validity, reliability, practicality, and washback. The study highlights the significance of the matriculation examination, evaluates individual test items, and presents the opinions of teachers and students about the test. The results reveal that…

Common test items play an important role in equating multiple test forms under the common-item nonequivalent groups design. Inconsistent item parameter estimates among common items can lead to large bias in equated scores for IRT true score equating. Current methods extensively focus on detection and elimination of outlying common items, which…

The computer-adaptive multistage testing (ca-MST) has been developed as an alternative to computerized adaptive testing (CAT), and been increasingly adopted in large-scale assessments. Current research and practice only focus on ca-MST panels for credentialing purposes. The ca-MST test mode, therefore, is designed to gauge a single scale. The…