This article presents key steps in the design and analysis of a computer based problem-solving assessment featuring interactive tasks. The purpose of the assessment is to support targeted instruction for students by diagnosing strengths and weaknesses at different stages of problem-solving. The first focus of this article is the task piloting…

Objective: Although health professionals increasingly are expected to be able to assess and manage patients' risk for suicide, few methods are available to evaluate this competency. This report describes development of a competency-assessment instrument for suicide risk-assessment (CAI-S), and evaluates its use in an objective structured clinical…

In this paper, we take a designer's look at how the activities and data of learning and assessment can be structured in immersive virtual game environments called Massively Multi-Player Online Games (MMOG). In doing so, we examine the channels of evidence through which learning and assessment activities are derived in MMOGs, offering examples of…

"WebGrader" is an online instrument developed to evaluate and assess oral competency in the basic course. It is the result of a 15-year study, using Morreale and Hackman's (1994) work as a general model, and "The Competent Speaker" speech rating criteria in particular (Morreale, Moore, Taylor, Surges-Tatum, & Hulbert-Johnson, 1990; Morreale,…

The purpose of this study was to develop and validate a content analysis model for assessing students' cognitive learning in asynchronous online discussions. It adopted a fully mixed methods design, in which qualitative and quantitative methods were employed sequentially for data analysis and interpretation. Specifically, the design was a…

Earlier studies have suggested that higher education institutions could harness the predictive power of Learning Management System (LMS) data to develop reporting tools that identify at-risk students and allow for more timely pedagogical interventions. This paper confirms and extends this proposition by providing data from an international…

This case study examined the educational software evaluation processes of pre-service teachers who attended either expertise-based training (XBT) or traditional training in conjunction with a Software-Evaluation checklist. Forty-three mathematics teacher candidates and three experts participated in the study. All participants evaluated educational…

This study examines the use of Internet online polling at a suburban junior high school as a means for decision makers to systematically acquire student perceptions about local conditions of learning. Federal and state mandates result in curricular, instructional, evaluative, and organizational changes that affect students' conditions of learning.…

This paper reports on a study of the use of self-repair among learners of German in a task-based CMC environment. The purpose of the study was two-fold. The first goal sought to establish how potential interpretations of CMC data may be very different depending on the method of data collection and evaluation employed. The second goal was to…

As course management systems (CMS) gain popularity in facilitating teaching. A forum is a key component to facilitate the interactions among students and teachers. Content analysis is the most popular way to study a discussion forum. But content analysis is a human labor intensity process; for example, the coding process relies heavily on manual…

Web server logs contain a great deal of information about who uses a Web site and how they use it. This article discusses the analysis of Web logs for instructional Web sites; reviews the data stored in most Web server logs; demonstrates what further information can be gleaned from the logs; and discusses analyzing that information for the…

Audit trail analysis can provide valuable insights to researchers and evaluators interested in comparing and contrasting designers' expectations of use and students' actual patterns of use of educational technology environments (ETEs). Sequence analysis techniques are particularly effective but have been neglected to some extent because of real…

Ethnography may be used to inform the audience of a curriculum program with the activities of the educators and students involved. Thus the expository function of evaluation is best met by ethnography. A conceptual model developed at the Central Midwestern Regional Educational Laboratory incorporates ethnographic data and techniques into…

Discusses the use and advantages of integrated learning systems (ILSs) in elementary and middle grades, and reanalyzes 30 evaluations of ILSs by using effect size statistics. Research problems are described, including poor evaluation designs and inadequate data collection and analysis; and a table summarizes the ILS research reports and…

Describes the development of a science-based computer simulation for sixth graders on global warming that integrated research, writing, speaking, science lab, and computer skills. Highlights include teacher/librarian collaboration; problem-based learning; building knowledge; design and planning; creating Web pages; gathering, organizing, and…