ERIC Number: ED672695
Record Type: Non-Journal
Publication Date: 2025-Jan
Pages: 25
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: 0000-00-00
Embedding Embedded Standard Setting: An Application of Cross-Classified Item Response Theory. CRESST Report 876
Yun-Kyung Kim; Li Cai
National Center for Research on Evaluation, Standards, and Student Testing (CRESST)
This paper introduces an application of cross-classified item response theory (IRT) modeling to an assessment utilizing the embedded standard setting (ESS) method (Lewis & Cook). The cross-classified IRT model is used to treat both item and person effects as random, where the item effects are regressed on the target performance levels (target PLs) assigned by item writers. The resulting regression coefficients reflect the alignment between the target PLs and item difficulty levels, a critical factor in evaluating the efficacy of the ESS method. A simulation study confirmed a successful recovery of the regression coefficients. We applied the model to an alternate English language proficiency assessment (Alt ELPA) developed for English learners with the most significant cognitive disabilities. The empirical example showcases the model's ability to provide validity and technical adequacy information for an assessment program serving a particularly small population, under heavy constraints on data collection design and reporting requirements.
Descriptors: Standard Setting (Scoring), Item Response Theory, Test Items, Difficulty Level, Alternative Assessment, English Learners
National Center for Research on Evaluation, Standards, and Student Testing (CRESST). 300 Charles E Young Drive N, GSE&IS Building 3rd Floor, Mailbox 951522, Los Angeles, CA 90095-1522. Tel: 310-206-1532; Fax: 310-825-3883; Web site: http://www.cresst.org
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Department of Education (ED)
Authoring Institution: National Center for Research on Evaluation, Standards, and Student Testing (CRESST)
Grant or Contract Numbers: S368A190007
Department of Education Funded: Yes
Author Affiliations: N/A