This study is an item analysis of the Matching Familiar Figures Test-20. We examined error scores in the Matching Familiar Figures Test-20 to determine the influence of the difficulty of the test on the assessment of reflection-impulsivity. The sample included 700 participants aged between 6 and 12 years. The results obtained from the corrected…

The current study examined the validation and alignment of the PASA-Science by determining whether the alternate science assessment anchors linked to the regular education science anchors; whether the PASA-Science assessment items are science; whether the PASA-Science assessment items linked to the alternate science eligible content, and what…

With the increasing popularity of peer assessment as an assessment tool, questions may arise about its measurement quality. Among such questions, the extent peer assessment practices adhere to standards of measurement. It has been claimed that new forms of assessment, require new criteria to judge their validity and reliability, since they aim for…

In this study, we describe a framework for monitoring rater performance over time. We present several statistical indices to identify raters whose standards drift and explain how to use those indices operationally. To illustrate the use of the framework, we analyzed rating data from the 2002 Advanced Placement English Literature and Composition…

This research is part of a larger study of commercial reading programs used in Canada in grades 1-6. The specific purposes of the results reported here were to identify and quantify the assessment techniques suggested for the selections that contain scientific content, to show how the assessments differ by grade, to evaluate the nature and quality…

This paper examines the Programme for International Student Assessment (PISA), which has become a major and influential component of the Organisation for Economic Cooperation and Development's (OECD) educational work. This measure of comparative performance of educational systems of member and other nations is based on tests commissioned by the…

The purposes of metaevaluation go beyond the traditional functions of accountability and enhancement. It helps guide strategic organizational change and legitimizes evaluation systems. Metaevaluation results can also be used to create checklists so that the persons responsible for any evaluation can revise, monitor, and control them by themselves.…

Combining diverse methods in a single study raises a problem: What should be done when the findings of one method of investigation conflict with those of another? The authors illustrate this problem using an example in which three study phases--quantitative, qualitative, and intervention--are applied. The findings from the quantitative phase did…

First, where we have been regarding measurement research will be discussed since the type, quality, and quantity of measurement research conducted in the past influences the type of measurement research which is presently conducted. Second, where we are now regarding measurement research will be discussed since the type of measurement research…

Lissitz and Samuelsen (2007) argue that the unitary conception of validity for educational assessments is too broad to guide applied work. They call for attention to considerations and procedures that focus on "test development and analysis of the test itself" and propose that those activities be collectively termed "content validity." The author…

The authors present a methodological review of empirical program evaluation research in the area of intimate partner violence prevention. The authors adapted and utilized criterion-based rating forms to standardize the evaluation of the methodological strengths and weaknesses of each study. The findings indicate that the limited amount of…

In this issue's "Ethical Challenges" column, commentators share their views of how evaluators might respond to specific problematic situations, linking their analyses to the principles and standards they believe are most relevant to the case. Not surprisingly, the perspectives that commentators offer are not always in agreement. When…

This paper integrates the emerging literatures of empirical research on the Community of Inquiry (CoI) framework and disciplinary effects in online teaching and learning by examining the disciplinary differences in perceptions of social, teaching, and cognitive presence of over 1500 students in seven disciplines at two U.S. institutions. Our…

Feedback on assessment tasks has an important part to play in underpinning student learning. Online assessment enables instantaneous feedback to be given so that the student can act on it immediately. However, concern has been expressed that e-assessment tasks (especially multiple-choice questions) can encourage surface-learning. Several projects…

In recent years there has been an increasing international interest in fine-grained diagnostic inferences on multiple skills for formative purposes. A successful provision of such inferences that support meaningful instructional decision-making requires (a) careful diagnostic assessment design coupled with (b) empirical support for the structure…