A technique is presented for applying the Rule Space methodology of cognitive diagnosis to assessment in a semantically rich domain. The approach bases diagnosis on item characteristics that are more abstract than individual problem-solving steps. The method is illustrated through a test of architectural knowledge completed by 122 architects. (SLD)

Item level differences between a type of constructed response item (figural response) and comparable multiple choice items in the domain of architecture were studied. Data from 120 architects and architecture students show that item level differences in difficulty correspond to differences in cognitive processing requirements and that relations…

Contrasts between constructed response items and stem-equivalent multiple-choice counterparts typically have involved averaging item characteristics, and this aggregation has masked differences in statistical properties at the item level. Moreover, even aggregated format differences have not been explained in terms of differential cognitive…

New developments in the use of automatically scorable constructed response item types for large-scale assessment are reviewed for five domains: (1) mathematical reasoning; (2) algebra problem solving; (3) computer science; (4) architecture; and (5) natural language. Ways in which these technologies are likely to shape testing are considered. (SLD)