A technique is presented for applying the Rule Space methodology of cognitive diagnosis to assessment in a semantically rich domain. The approach bases diagnosis on item characteristics that are more abstract than individual problem-solving steps. The method is illustrated through a test of architectural knowledge completed by 122 architects. (SLD)

Item level differences between a type of constructed response item (figural response) and comparable multiple choice items in the domain of architecture were studied. Data from 120 architects and architecture students show that item level differences in difficulty correspond to differences in cognitive processing requirements and that relations…

Contrasts between constructed response items and stem-equivalent multiple-choice counterparts typically have involved averaging item characteristics, and this aggregation has masked differences in statistical properties at the item level. Moreover, even aggregated format differences have not been explained in terms of differential cognitive…

This paper presents a technique for applying the Rule Space Model of cognitive diagnosis (Tatsuoka, 1983) to assessment in a semantically rich domain. Responses of 122 architects to 22 architecture test items developed to assess a range of architectural knowledge were analyzed using Rule Space. Verbal protocol analysis guided the construction of a…