Logic modeling, the process that explicates how programs are constructed and theorized to bring about change, is considered to be standard evaluation practice. However, logic modeling is often experienced as a transactional, jargon-laden, discrete task undertaken to produce a document to comply with the expectations of an external entity, the…

This paper is concerned with the process of selecting between the increasingly popular bi-factor model and the second-order factor model in measurement research. It is indicated that in certain settings widely used in empirical studies, the second-order model is nested in the bi-factor model and obtained from the latter after imposing appropriate…

The population relationship between coefficient alpha and scale reliability is studied in the widely used setting of unidimensional multicomponent measuring instruments. It is demonstrated that for any set of component loadings on the common factor, regardless of the extent of their inequality, the discrepancy between alpha and reliability can be…

Various areas of educational research are interested in the transitions between different states--or events--in sequential data, with the goal of understanding the significance of these transitions; one notable example is affect dynamics, which aims to identify important transitions between affective states. Unfortunately, several works have…

Evaluative criteria define a "high quality" or "successful" evaluand and provide the basis for judgment of merit and worth, yet they are often assumed and implicit in the evaluation process. This article presents an empirically supported model that describes and integrates two aspects of criteria: domain and source. Domain…

Despite the rising popularity of big data, there is speculation that evaluators have been slow adopters of these new statistical approaches. Several possible reasons have been offered for why this is the case: ethical concerns, institutional capacity, and evaluator capacity and values. In this method note, we address one of these barriers and aim…

Premised on the idea that evaluators should be familiar with a range of approaches to program modifications, I review several existing approaches and then describe another, less well-recognized option. In this newer option, evaluators work with others to identify potentially needed adaptations for select program aspects "in advance." In…

This article explores the models and frameworks developed on "research impact'. We aim to provide a comprehensive overview of related literature through scoping study method. The present research investigates the nature, objectives, approaches, and other main attributes of the research impact models. It examines to analyze and classify models…

Although the literature recognizes training as an essential driver of organizational effectiveness, little is still known about how to explicitly focus and align training to organizational strategic priorities. This note proposes a model that bridges the strategic human resource management (SHRM) literature and the Balanced Scorecard (BSC)…

In an age of accountability, numbers matter. Therefore, the accountability movement has led to the assessment movement. Higher education should be held accountable, and colleges themselves should take the lead in making sure colleges are meeting goals. Assessment was originally meant to improve higher education. Instead, assessment quickly became…

Early investigation led the Evaluative Study of Action Research (ESAR) team to conclude that the complexity of a global, large scale study (evaluation of more than 100 highly diverse action research [AR] projects) called for an overarching research evaluation framework that differed from traditional frameworks. This article details the flexible,…

The researchers of various disciplines often use qualitative and quantitative research methods and approaches for their studies. Some of these researchers like to be known as qualitative researchers; others like to be regarded as quantitative researchers. The researchers, thus, are sharply polarised; and they involve in a competition of pointing…

The examinee-selected-item (ESI) design, in which examinees are required to respond to a fixed number of items in a given set of items (e.g., choose one item to respond from a pair of items), always yields incomplete data (i.e., only the selected items are answered and the others have missing data) that are likely nonignorable. Therefore, using…

Several large scale assessments include student, teacher, and school background questionnaires. Results from such questionnaires can be reported for each item separately, or as indices based on aggregation of multiple items into a scale. Interpreting scale scores is not always an easy task though. In disseminating results of achievement tests, one…

There is general consensus that teaching and learning centres have a positive impact on promoting and supporting a culture that values teaching and learning, yet there is limited evidence on how centres evaluate their work. For this purpose, a survey was developed and administered to the directors of 88 Canadian centres; 46 of which completed the…