Model evaluation -- the process of making inferences about the performance of predictive models -- is a critical component of predictive modelling research in learning analytics. We survey the state of the practice with respect to model evaluation in learning analytics, which overwhelmingly uses only naïve methods for model evaluation or…

Research on the evaluation of large-scale public-sector reforms is rare. This article sets out to fill that gap in the evaluation literature and argues that it is of vital importance since the impact of such reforms is considerable and they change the context in which evaluations of other and more delimited policy areas take place. In our…

This "What Works Clearinghouse Procedures and Standards Handbook (Version 3.0)" provides a detailed description of the standards and procedures of the What Works Clearinghouse (WWC). The remaining chapters of this Handbook are organized to take the reader through the basic steps that the WWC uses to develop a review protocol, identify…

This study investigated the use of a contextually sensitive instrument to assess the effect of invasive species monitoring training on the scientific literacy of citizen volunteers. The authors measured scientific literacy scores before and after 57 citizens participated in a 2-day event to learn to monitor invasive species with an instrument…

The main thesis of this paper is that testing is now ready for a major effort to create a synthesis out of what has hitherto been a series of unrelated approaches to testing. The paper proposes the methods of Measurement, Evaluation and Assessment as three very different approaches to the problem of testing, and discusses their characteristics. It…

The 2003 NAAL is a complex assessment with several components and various types of data. The primary purpose of this publication is to describe the assessment's key features and data types. Thus, the publication covers the critical concepts and features carried over from the 1992 assessment, as well as those new to the 2003 assessment--for…

Evaluates "Booklist,""Choice,""Library Journal," and "the New York Times Book Review" by measuring and comparing (1) total number of adult books reviewed, (2) total number of juvenile and young adult books reviewed, (3) percentage of randomly selected Notable Books reviewed, and (4) how quickly those Notable Books were reviewed. (Author)

Compares "Book Review Digest,""Book Review Index," and "Current Book Review Citations" for their coverage of titles appearing on the best-seller lists of "Publishers Weekly." Methodology is described and results are provided for both 1976 and 1978 titles. (Author/JD)

Both goal-free and goal-based methodologies were operationalized. Subjects for the study were nominated by leaders in the evaluation field. Subjects were randomly selected and assigned to treatments. Sites were randomly assigned to either methodology. Reports generated from each were sent to project directors to be rated on criteria found in the…

Examines four approaches to evaluating writing, and notes the strengths and weaknesses of each. (RL)

Students reading the experimental handbook took less time, demonstrated more attending behavior, and made more favorable comments. Their scores on an information test indicated it was more effective than the traditional handbook. Poor readers in this group did better than poor readers in the control group. This study emphasizes systematic…

Ten computer languages are compared and contrasted. A series of programs focusing on the average of two numbers are designed to point out differences in console input, console output, and common arithmetic. (MP)

A comparison of research and development and teacher competency assessment activities is divided into two tasks: (1) selection and cataloging of competencies; and (2) development of instruments and procedures to assess competencies for diagnostic training and selection. Similarities and differences in selecting and cataloging teacher competencies…

Presents a conceptual framework for comparing instructional design models and employing appropriate design applications. Highlights include the classification system on which the framework is centered and how the framework allows for the models' instructional context and the size of the audience. (27 references) (KRN)

This book presents seven essays on evaluation in higher education. In "Threats and Opportunities of Evaluation in Higher Education" Roeland in't Veld sets the policy frameworks for increased emphasis on evaluation. In "Institutional Evaluation: The Role of the Main Actors in Higher Education" Andre Staropoli describes the roles of the forces which…