Social networking applications such as WhatsApp have been extensively used for language research; however, they have rarely been applied for language assessment purposes. To explore the efficiency of WhatsApp for assessment purposes, 30 Iranian English learners doing self- and peer-assessments on WhatsApp are studied. The changes and the reasons…

This study used think-aloud protocols to compare essay-rating processes across holistic and analytic rating scales in the context of China's College English Test Band 6 (CET-6). A group of 9 experienced CET-6 raters scored the same batch of 10 CET-6 essays produced in an operational CET-6 administration twice, using both the CET-6 holistic…

Assessing pragmatic knowledge in the instructed setting is seen as a complex but necessary task, which requires the design of appropriate research methodologies to examine pragmatic performance. This study discusses the use of two different research methodologies, namely those of Discourse Completion Tests/Tasks (DCTs) and verbal reports. Research…

This article discusses measuring learning strategies by means of questionnaires. In "multi-method" research, in which think-aloud measures are compared with questionnaires, low or moderate correlations are found. A conclusion often drawn is that learners are not able to verbally report on their learning activities. Alternative explanations concern…

Findings derived from self-reported, structured survey questionnaires are commonly used in evaluation and applied research to inform policy-making and program decisions. Although there are a variety of issues related to the quality of survey evidence (e.g., sampling precision), the validity of response processes--how respondents process thoughts…

Self-regulated learning has been described as an adaptive process: students adapt their learning strategies for attaining different learning goals. In order to be adaptive, students must have a clear notion of what the task requirements consist of. Both trace data and questionnaire data indicate that students adapt study strategies in limited ways…

Third grade elementary school children solved tests on mathematical reasoning and numerical facility. Metacognitive skillfulness was assessed through think aloud protocols, prospective and retrospective child ratings, teacher questionnaires, calibration measures and EPA2000. In our dataset metacognition has a lot in common with intelligence, but…

The topic of how students become self-regulated as learners has attracted researchers for decades. Initial attempts to measure self-regulated learning (SRL) using questionnaires and interviews were successful in demonstrating significant predictions of students' academic outcomes. The present article describes the second wave of research, which…

While different approaches to evaluation will yield different results, depending on the purpose of the evaluation this article describes an evaluation approach that was aimed at investigation of the mental models of users of the programme. The study was driven by questions about the differences in mental models of the instructional designers and…

Discusses advantages and limitations of six usability evaluation methods (heuristic evaluation, pluralistic walk-throughs, formal usability inspections, empirical methods, cognitive walk-throughs, and formal design analysis) and six data collection techniques (verbal reports, concurrent think-aloud method, questionnaire, video analysis,…

Introduction: Despite all the emphasis on metacognition, researchers currently use different techniques to assess metacognition. The purpose of this contribution is to help to clarify some of the paradigms on the evaluation of metacognition. In addition the paper reviews studies aiming to improve the learning process through metacognition. Method:…

Purpose: We developed and evaluated an explicit procedure for obtaining self-report pain data from nursing home residents across a broad range of cognitive status, and we evaluated the consistency, stability, and concurrent validity of resident responses. Design and Methods: Using a modification of the Geriatric Pain Measure (GPM-M2), we…

Effective evaluation of educational software is a key issue for successful introduction of advanced tools in the curriculum. This paper details to developing and evaluating a tool for computer assisted learning of science laboratory courses. The process was based on the generic instructional system design model. Various categories of educational…

Generally, experts agree on what good readers do to comprehend text--they connect new text with past experiences, interpret, evaluate, synthesize, and consider alternative interpretations. Yet, traditional measures of reading comprehension only provide a general indicator of how well a student understands text. They do not provide information…