This report provides an in-depth comparison of TOEFL iBT® and the Duolingo English Test (DET) in terms of the degree to which both tests assess academic language proficiency in listening, reading, writing, and speaking. The analysis is based on publicly available documentation on both tests, including sample test questions available on the test…

This demonstration introduces and presents an innovative online cognitive diagnostic assessment, developed to identify the types of cognitive processes that readers use during comprehension; specifically, processes that distinguish between subtypes of struggling comprehenders. Cognitive diagnostic assessments are designed to provide valuable…

Our aim was to deepen understanding of public examinations, exploring how marking task demands influence examiners' cognition and ultimately their marking accuracy. To do this, we identified features of examinations that trigger or demand the use of cognitive marking strategies entailing "reflective" judgements. Kelly's Repertory Grid…

Don't settle for assessing recall and comprehension only when you can use this guide to create assessments for higher-order thinking skills. Assessment expert Susan M. Brookhart brings you up to speed on how to develop and use test questions and other assessments that reveal how well your students can analyze, reason, solve problems, and think…

Due to the lack of span test for the use in language-specific and cross-language studies, this study provides L1 and L2 researchers with a reliable language-independent span test (math span test) for the measurement of working memory capacity. It also describes the development, validation, and scoring method of this test. This test included 70…

It has long been established that marking accuracy in public examinations varies considerably among subjects and markers. This is unsurprising, given the diverse cognitive strategies that the marking process can entail, but what makes some questions harder to mark accurately than others? Are there distinct but subtle features of questions and…

Only a small number of qualitative studies have investigated panelists' experiences during standard-setting activities or the thought processes associated with panelists' actions. This qualitative study involved an examination of the experiences of 11 panelists who participated in a prior, one-day standard-setting meeting in which either the…

This paper describes a work in progress on a second edition of "Taxonomy of Educational Objectives, The Classification of Educational Goals, Handbook I: Cognitive Domain," also known as "Bloom's Taxonomy" (B. Bloom and others, Eds., 1956). The new edition will be grounded in the collective wisdom of the original…

Conjunctive item response models are introduced such that: (1) sufficient statistics for latent traits are not necessarily additive in item scores; (2) items are not necessarily locally independent; and (3) existing compensatory (additive) item response models including the binomial, Rasch, logistic, and general locally independent model are…

In some settings, the validity of a battery composite or a test score is enhanced by weighting some parts or items more heavily than others in the total score. This article describes methods of estimating the total score reliability coefficient when differential weights are used with items or parts.

Constructed-response formats are desired for measuring complex and dynamic response processes that require the examinee to understand the structures of problems and micro-level cognitive tasks. These micro-level tasks and their organized structures are usually unobservable. This study shows that elementary graph theory is useful for organizing…

The three-parameter logistic model discussed was used by CTB/McGraw-Hill in the development of the Comprehensive Tests of Basic Skills, Form U (CTBS/U) and the Test of Cognitive Skills (TCS), published in the fall of 1981. The development, standardization, and scoring of the tests are described, particularly as these procedures were influenced by…

A new standard setting approach is introduced, called the cognitive components approach. Like the Angoff method, the cognitive components method generates minimum pass levels (MPLs) for each item. In both approaches, the item MPLs are summed for each judge, then averaged across judges to yield the standard. In the cognitive components approach,…

The cognitive components standard setting method, recently introduced by D. McGinty and J. Neel (1996), asks judges to specify minimum levels of performance not for the test items, but for smaller portions of items, the component skills and concepts required to answer each item correctly. Items are decomposed into these components before judges…

The purpose of this essay is to explore both what is known and what needs to be learned about the information value of performance items "when they are used in large scale assessments." Within the context of the National Assessment of Educational Progress (NAEP), there is substantial motivation for answering these questions. Over the…