Logic modeling, the process that explicates how programs are constructed and theorized to bring about change, is considered to be standard evaluation practice. However, logic modeling is often experienced as a transactional, jargon-laden, discrete task undertaken to produce a document to comply with the expectations of an external entity, the…

Most program evaluation efforts concentrate on assessments of "program implementation" and "program outcomes." However, another area of programs that has not received sufficient attention in the literature is evaluating the plan of the program. Since the quality of the plan and planning process can influence program…

Premised on the idea that evaluators should be familiar with a range of approaches to program modifications, I review several existing approaches and then describe another, less well-recognized option. In this newer option, evaluators work with others to identify potentially needed adaptations for select program aspects "in advance." In…

Over the last few decades, there has been an increase in the number of large federally funded transdisciplinary programs and initiatives. Scholars have identified a need to develop frameworks, methodologies, and tools to evaluate the effectiveness of these large collaborative initiatives, providing precise ways to understand and assess the…

Social Network Analysis (SNA) is a statistical method used to analyze the social structure and interactions among individuals within a network. SNA is used extensively in a number of disciplines such as sociology, geography, and communications research. However, the use of SNA by practitioners and researchers in assessment and evaluation is much…

The University English Centres of Australia (UECA) 'Assessment Symposium: Assessment Practice in a Changing World' at Southern Cross University in October 2022 highlighted the growing interest in integrated assessment in Australian university direct entry programs (DEPs) for English as Additional Language (EAL) students. In keynote presentations,…

The issue of connecting research evidence with practice referred to as evidence-informed practice (EIP) has received considerable critical attention in the field of education. However, assessment literature has failed to fully acknowledge the significance of this issue. This conceptual paper concentrates on the process of informing classroom…

The European University Association (EUA) has continuously supported European universities in the transition towards Open Science and, in particular, to Open Access. Upon recommendation of its Expert Group on Science 2.0/Open Science, EUA has developed a variety of initiatives in this area, as outlined in the "EUA Roadmap on Open Access to…

In an age of accountability, numbers matter. Therefore, the accountability movement has led to the assessment movement. Higher education should be held accountable, and colleges themselves should take the lead in making sure colleges are meeting goals. Assessment was originally meant to improve higher education. Instead, assessment quickly became…

Many studies of education interventions make claims about impacts on students' outcomes. Some studies have designs that enable readers to make causal inferences about the effects of an intervention but others have designs that do not permit these types of conclusions. To help policymakers, practitioners, and others make sense of study results, the…

Evaluation of professional learning illuminates the interactions that occur in the implementation of planned learning experiences and the necessary supports designed to improve professional practice and its effects on students. It investigates how a set of actions designed to achieve defined short- and long-term outcomes occur over time and how…

Construct irrelevance (CI) and construct underrepresentation (CU) are 2 major threats to validity, yet they are rarely discussed within the counseling literature. This article provides information about the relevance of these threats to internal validity. An illustrative case example will be provided to assist counselors in understanding these…

Precision is a key facet of test development, with score reliability determined primarily according to the types of error one wants to approximate and demonstrate. This article identifies and discusses several primary forms of reliability estimation: internal consistency (i.e., split-half, KR-20, a), test-retest, alternate forms, interscorer, and…

The What Works Clearinghouse (WWC) evaluates research studies that look at the effectiveness of education programs, products, practices, and policies, which the WWC calls "interventions." Many studies of education interventions make claims about impacts on students' outcomes. Some studies have designs that enable readers to make causal…

Like instruments used in afterschool programs to assess children's social and emotional growth or to evaluate staff members' performance, instruments used to evaluate program quality should be free from bias. Practitioners and researchers alike want to know that assessment instruments, whatever their type or intent, treat all people fairly and do…