Background/Context: Principles of evidential reasoning have often been discussed in the context of educational and psychological measurement with respect to construct validity and validity arguments. More recently, Mislevy proposed the metaphor of assessment as an evidentiary argument about students' learning and abilities given their…

This dissertation reports the results of three studies and a pilot study. The first study was a Monte Carlo validation study that examined the accuracy of a new visual inspection method, the semi-interquartile range method. Results of the study indicated that this method had lower levels of power than a previously validated method, the…

Educational evaluation in too many states and school districts, if done at all across an institution, has been reduced to crunching test scores in only the subject areas and grade levels tested. While measuring student achievement in reliable and valid ways is an important part of the mission of education, it is not enough. What's also needed is a…

Purpose: District effectiveness research (DER) is an emerging field concerned with identifying the organizational structures, administration, and leadership practices at the school district level that help districts find success with all of their students across the schools within the system. This work has mirrored much of the early school…

This research investigates the effect of a web-based model, named "Practicing, Reflecting, and Revising with Web-based Assessment and Test Analysis system (P2R-WATA) Assessment Literacy Development Model," on enhancing assessment knowledge and perspectives of secondary in-service teachers, and adopts a single group experimental research…

In this chapter, the author proposes a model for assessing institutional effectiveness. The Volkwein model for assessing institutional effectiveness consists of five parts that summarize the steps for assessing institutions, programs, faculty, and students. The first step in the model distinguishes the dual purposes of institutional effectiveness:…

Evaluative statements from alumni have legitimacy with both internal and external stakeholders. Thus, alumni studies can provide valuable evidence of institutional effectiveness and lend themselves to both locally designed and commercial survey instruments for data collection. Under ideal conditions, the results of alumni studies equip faculty and…

This guide is designed to help districts and schools evaluate reform efforts related to the American Recovery and Reinvestment Act (ARRA), including School Improvement Grants. It includes practical, how-to tips for conducting program evaluations, whether one takes a "do-it-yourself" approach or seeks outside assistance to evaluate reform…

In current psychological research, the analysis of data from computer-based assessments or experiments is often confined to accuracy scores. Response times, although being an important source of additional information, are either neglected or analyzed separately. In this article, a new model is developed that allows the simultaneous analysis of…

An assessment framework is like a blueprint, laying out the basic design of the assessment by describing the mathematics content that should be tested and the types of assessment questions that should be included. It also describes how the various design factors should be balanced across the assessment. This is an assessment framework, not a…

Evaluations of practice have often adopted experimental or quasi-experimental approaches and have focused heavily on outcomes of innovative programmes. This creates problems in replicating studies and effectively creates obstacles to the sharing of good practice. In this paper, ideas from Pawson and Tilley's (1997) approach to Realistic Evaluation…

In response to recent trends and legislation, the concept of implementing evidence-based practices has become a critical component of contemporary schooling. It is important that teachers and families of students with disabilities understand the role that qualitative research plays in determining whether a practice is in fact evidence based.…

This paper helps researchers develop criteria for conducting surveys. Cost considerations frame the paper's suggestions. Specifically, the text advises that researchers consider the human and material resources needed, type of survey instrument, medium for conducting the survey (mail, telephone, personal interview, etc.), data analysis…

Important findings are often a balance between the rigor of the experimental design and innovativeness of the experimental question. One broad topic area that has received a great deal of discussion, but little empirical study, is the evaluation of educational systems. Experimental designs that permit the analysis of practices used by state…

Evaluation studies on educational questions attempt to provide answers in the form of conclusions or inferences which are derived from the information collected. Valid inferences are a result of careful research design for the study, and may be causal, descriptive, value-oriented, or probabilistic. Basic steps in designing an evaluation study are…