There is a need for assessment items that assess complex constructs but can also be efficiently scored for evaluation of teacher education programs. In an effort to measure the construct of teacher attentiveness in an efficient and scalable manner, we are using exemplar responses elicited by constructed-response item prompts to develop…

Assessments are usually thought of as ways for instructors to get information from students. In this work, we flip this perspective and explore how assessments communicate information to students. Specifically, we consider how assessments may provide information about what faculty and/or researchers think it means to know and do physics, i.e.,…

Consistent with common core state standards and recent legislation such as the Every Student Succeeds Act, middle level teachers are establishing high expectations for all students. Middle level students are expected to make connections among topics, cite evidence from text, and engage in higher level thinking skills across subject areas. Often,…

Notwithstanding the introduction of education multilingual policies worldwide, testing and assessment procedures still rely almost exclusively on the monolingual construct. This paper describes a study, part of a larger project fostering a new multilingual education policy in Israeli schools, exploring bi/multilingual assessment. It included two…

The literature on C-test and Cloze test in a second language provides us with few accounts of the real mental processes that test-takers are involved in, which in fact indicate the real nature of what these tests measure. However, the literature leaves researchers with little attention to the cognitive processes involved in X-Tests. Therefore, the…

The computer-based "Graduate Record Examinations"® ("GRE"®) revised General Test includes interactive item types and testing environment tools (e.g., test navigation, on-screen calculator, and help). How well do test takers understand these innovations? If test takers do not understand the new item types, these innovations may…

This article considers the design process for new formative history assessments. Over the course of 3 years, my colleagues from the Stanford History Education Group and I designed, piloted, and revised dozens of "History Assessments of Thinking" (HATs). As we created HATs, we sought to gather information about their cognitive validity,…

Studied the cognitive equivalence of computerized and paper-and-pencil reading comprehension tests using verbal protocol analysis. Results for 48 college students indicate that the only significant difference between the computerized and paper-and-pencil tests was in the frequency of identifying important information in the passage. (SLD)

The comparability of computerized and paper-and-pencil tests was examined from cognitive perspective, using verbal protocols rather than psychometric methods, as the primary mode of inquiry. Reading comprehension items from the Graduate Record Examinations were completed by 48 college juniors and seniors, half of whom took the computerized test…

A study examined whether the process of gathering verbal reports of subjects' thinking while taking multiple-choice critical thinking tests could be used to infer the reasoning process used and identify test items which do not require critical thinking skills. Four factors can render an inference of a subject's critical thinking skills…

In response to the tradition of examining a new test's validity by comparing it with a well-established multiple-choice test, a study investigated whether multiple-choice tests with one right answer can measure the reading comprehension process as defined by constructivists. The study compared the information obtained from two different…

The recall protocol procedure can be used as an alternative measure of foreign language reading comprehension and as a diagnostic tool. Protocol use encourages students to attend to the meaning of a foreign language passage and helps teachers to instruct students in the complex art of meaning construction. (CB)

This study investigated and compared the effects of two test formats (free response and multiple choice) on English-as-a-Second-Language (ESL) learners' reading comprehension. The tests, together with a checklist of test-taking strategies and retrospective questionnaires concerning more general reading strategies, were administered to 57 ESL…

A method for understanding and controlling the multiple facets of an oral examination (OE) or other judge-intermediated examination is presented and illustrated. This study focused on determining the extent to which the facets model (FM) analysis constructs meaningful variables for each facet of an OE involving protocols, examiners, and…