Who should evaluate the originality and task-appropriateness of a given idea has been a perennial debate among psychologists of creativity. Here, we argue that the most relevant evaluator of a given idea depends crucially on the level of expertise of the person who generated it. To build this argument, we draw on two complimentary theoretical…

To address response style or bias in rating scales, forced-choice items are often used to request that respondents rank their attitudes or preferences among a limited set of options. The rating scales used by raters to render judgments on ratees' performance also contribute to rater bias or errors; consequently, forced-choice items have recently…

Automatic machine learning (AutoML) methods automate the time-consuming, feature-engineering process so that researchers produce accurate student models more quickly and easily. In this paper, we compare two AutoML feature engineering methods in the context of the National Assessment of Educational Progress (NAEP) data mining competition. The…

A quality management system is an essential and mandatory instrument for maritime higher education institutions mandated by the International Convention on Standards of Training, Certification, and Watchkeeping for Seafarers. To design an effective quality management system, it is strongly recommended to take the most comprehensive model of the…

Rating scales for writing assessment are critical in that they determine directly the quality and fairness of such performance tests. However, in many EFL contexts, rating scales are made, to certain extent, based on the intuition of teachers who strongly need a feasible and scientific route to guide their construction of rating scales. This study…

King Abdulaziz University (KAU) took extra measures in response to COVID-19 in March 2020. The current research evaluated an Emergency Training Session (ETS) Knowledge Model during the pandemic with the aim of identifying for the Deanship of e-Learning and Distance Education (DeLDE) to evaluate how the e-learning services support learning and…

This article provides results from a study of the hidden processes of consensus formation that precede and make possible official judgments and decisions of the Swedish Schools Inspectorate (SI). The research question for the study was: How is knowledge negotiated on the back stage of school inspection and presented on the front stage? The article…

This paper discusses two models that hold promise for helping evaluators understand and cope with different decision contexts: (1) the conflict Model (Janis and Mann, 1977) and the Social Process Model (Vroom and Yago, 1974). Implications and guidelines for using decisionmaking models in evaluation settings are presented. (BS)

The making of a decision implies a choice between competing alternative solutions. In a decision-making context including multiple criteria, the criteria (or their related variables) are placed in relation to each other. This provides a comparison to some fixed point or origin, reducing the problem to one which can be solved by optimization as in…

A 19-step general evaluation model is described through its four stages: identifying problems, prescribing program solutions, evaluating the operation of the program, and evaluating the effectiveness of the model. The role of the evaluator in decision making is also explored. (RAO)

Evaluators usually have preconceptions about how decisions are made in social programs and how evaluation results will be used. This article demonstrates how an evaluator's choice of a decision-making model has significant impact on the conduct and fate of the research. (Author/CM)

An ethnographic evaluation adapts data collection and analytical methods of the ethnographer to the purpose of decision-making. Ethnography and evaluation differ in determination of problems addressed by the researcher. Ethnographers are interested in slow cultural change, while evaluators address policy questions requiring immediate answers. The…

The literature on the use of evaluative information in local education agencies is summarized. A general structure for studying evaluation use is presented. It is organized after the commonplaces of education (teacher, student, and subject matter). (BW)

Practical suggestions to an evaluator are presented. It is seen as essential that an evaluator who would influence decision making through research realize that the research in which he engages, the data collected, and the analysis and interpretation are influenced by the values of the institution that the evaluator serves. (MLW)

This review of the state of educational evaluation, which is directed to both those who produce and those who consume evaluation data, suggests that certain prevalent evaluation practices are seriously out of step with, and inappropriate for, current mainstream instructional activity. Evaluation is recommended as not merely an appendage to…