Two of the central issues in comparative judgement (CJ), which are perhaps underexplored compared to questions of the method's reliability and technical quality, are "what processes do judges use to make their decisions" and "what features do they focus on when making their decisions?" This article discusses both, in the…

Notwithstanding the introduction of education multilingual policies worldwide, testing and assessment procedures still rely almost exclusively on the monolingual construct. This paper describes a study, part of a larger project fostering a new multilingual education policy in Israeli schools, exploring bi/multilingual assessment. It included two…

Nowadays, almost everyone uses the World Wide Web (WWW) to search for information of any kind. In education, students frequently use the WWW for selecting information to accomplish assignments such as writing an essay or preparing a presentation. The evaluation of sources and information is an important sub-skill in this process. But many students…

The authors describe several types of classroom assessment techniques (CATs) and cognitive scaffolding procedures that they have developed over the years. They then bring the procedures together in a sample lecture/group learning class presentation.

Even if national and international assessments are designed to be comparable, subsequent psychometric analyses often reveal differential item functioning (DIF). Central to achieving comparability is to examine the presence of DIF, and if DIF is found, to investigate its sources to ensure differentially functioning items that do not lead to bias.…

The important but little understood problem that motivated this study was the lack of research on valid assessment methods to determine progress in higher-order learning in situations involving complex and ill-structured problems. Without a valid assessment method, little progress can occur in instructional design research with regard to designing…

Instructors in electrical and computer engineering and in computer science have developed innovative methods to teach digital logic circuits. These methods attempt to increase student learning, satisfaction, and retention. Although there are readily accessible and accepted means for measuring satisfaction and retention, there are no widely…

Elementary general music teachers typically teach hundreds of students every week. Each child has individual learning needs due to a variety of factors, such as prior experiences with music, music aptitude, learning style, and personality. The purpose of this qualitative study was to explore ways that experienced teachers used assessments to…

An order form used in a university printing office was evaluated, redesigned, and tested to determine whether the original or the revised version could be completed more quickly and accurately. Two types of subjects were used: six "experienced" subjects who had previously used the old version of the form, and six "novice" subjects who had never…

Describes differences in formative evaluation techniques between instructional designers and subject matter experts. Content analysis based on think-aloud data focusing on differences in task interpretation, problem identification and revision statements, and review strategies is discussed; use of instructional systems design models by…

This study examined planning processes and the acquisition of expert problem solving skills among individuals of different expertise on tasks of increasing complexity. Twelve French-speaking adult students of machining (novices) and 12 French-speaking teachers of machining (experts) in Quebec (Canada) were randomly assigned to a simple or complex…

In response to the tradition of examining a new test's validity by comparing it with a well-established multiple-choice test, a study investigated whether multiple-choice tests with one right answer can measure the reading comprehension process as defined by constructivists. The study compared the information obtained from two different…

Whether high levels of metacognitive knowledge about problem solving can compensate for low overall aptitude was studied for 31 high-aptitude and 25 low-aptitude children in grades 4 and 5. Results confirming superior performance of high metacognitive individuals are discussed in terms of the independence of metacognition and academic aptitude.…

The methodology developed for the Western Cooperative for Educational Telecommunications' evaluation of the "New Pathways to a Degree" projects funded by the Annenberg/CPB Project is described. This methodology provides an approach for conducting a valid, reliable evaluation effort that is sensitive to specific institutional issues and…

How people evaluate primary and secondary sources when considering historical questions was studied by asking eight historians and eight high school seniors to think aloud while reviewing written and pictorial documents about the Battle of Lexington. Differences between the groups are discussed concerning cognitive processes and beliefs about…