The purpose of this study is to generate non-verbal items for a visual reasoning test using templated-based automatic item generation (AIG). The fundamental research method involved following the three stages of template-based AIG. An item from the 2016 4th-grade entrance exam of the Science and Art Center (known as BILSEM) was chosen as the…

This study reports on an Evidence Centered Design (ECD) project in the Netherlands, involving the theory exam for prospective car drivers. In particular, we illustrate how cognitive load theory, task-analysis, response process models, and explanatory item-response theory can be used to systematically develop and refine task models. Based on a…

In calculus courses, instructors often use the end-of-section problems in a textbook in homework assignments or other course assessments. As a result, these problems influence the teaching and learning of calculus. In this study, we examine the levels of cognitive demand of these problems in a mainstream calculus textbook and classify them within…

Despite being a popular topic in language testing, cognitive load has not received enough attention in vocabulary test items. The purpose of the current study was to scrutinize the cognitive load and vocabulary test items' differences, examinees' reaction times, and perceived difficulty. To this end, 150 students were selected using…

The use of computer-based tests (CBTs), for both formative and summative purposes, has greatly increased over the past years. One major advantage of CBTs is the easy integration of multimedia. It is unclear, though, how to design such CBT environments with multimedia. The purpose of the current study was to examine whether guidelines for designing…

The objective structural practical examination (OSPE) is a timed examination that assesses topographical and/or applied knowledge of anatomy with the use of cadaveric resources and medical images. This study investigated whether elements of question design (provision of clinical context, type of visual resources used, gender context, and…

As more instructors articulate learning objectives for their students within one course, or academic staff collaborate to articulate learning outcomes for programs, a robust means to assess student performance within these becomes increasingly important. The Examinations Institute of the American Chemical Society (ACS), Division of Chemical…

In this paper we present a model for computerized adaptive practice and monitoring. This model is used in the Maths Garden, a web-based monitoring system, which includes a challenging web environment for children to practice arithmetic. Using a new item response model based on the Elo (1978) rating system and an explicit scoring rule, estimates of…

This study investigated how the format of verbal instructions in computer simulations and prior knowledge (PK) affected 8th graders' cognitive load (CL) level and achievement in a multimedia learning environment. Although PK was not found to significantly affect student performance and CL level, instruction format was found to impact both.…

The purpose of the study reported herein was to identify the common mistakes made by junior secondary students in Hong Kong when learning algebra and to compare teachers' perceptions of students' ability with the results of an algebra test. An algebra test was developed and administered to a sample of students (aged between 13 and 14 years). From…

For educational achievement tests that employ multiple-choice (MC) items and aim to reliably classify students into performance categories, it is critical to design MC items that are capable of discriminating student performance according to the stated achievement levels. This is accomplished, in part, by clearly understanding how item design…