The What Works Clearinghouse (WWC) evaluates research studies that look at the effectiveness of education programs, products, practices, and policies, which the WWC calls "interventions." Many studies of education interventions make claims about impacts on students' outcomes. Some studies have designs that enable readers to make causal…

Like instruments used in afterschool programs to assess children's social and emotional growth or to evaluate staff members' performance, instruments used to evaluate program quality should be free from bias. Practitioners and researchers alike want to know that assessment instruments, whatever their type or intent, treat all people fairly and do…

Although there are tools to assess student's readiness in an "online learning context," little is known about the "psychometric" properties of the tools used or not. A systematic review of 5107 published and unpublished papers identified in a literature search on student online readiness assessment tools between 1990 and…

The absence of universally accepted definitions for direct and indirect assessment motivates the purpose of this article: to offer definitions that are literature-based and theoretically driven, meeting K. Lewin's (1945) dictum that, "There is nothing so practical as a good theory" (p. 129). The authors synthesize the literature to…

Background/Context: The target of assessment validation is not an assessment but the use of an assessment for a purpose. Although the validation literature often provides examples of assessment purposes, comprehensive reviews of these purposes are rare. Additionally, assessment purposes posed for validation are generally described as discrete and…

Any examination that involves moderate to high stakes implications for examinees should be psychometrically sound and legally defensible. Currently, there are two broad and competing families of test theories that are used to score examination data. The majority of instructors outside the high-stakes testing arena rely on classical test theory…

Evaluators have an obligation to present clearly the results of their evaluative efforts. Traditionally, such presentations showcase formal written and oral reports, with dispassionate language and graphs, tables, quotes, and vignettes. These traditional forms do not reach all audiences nor are they likely to include the most powerful presentation…

School climate surveys are widely applied in school districts across the nation to collect information about teacher efficacy, principal leadership, school safety, students' activities, and so forth. They enable school administrators to understand and address many issues on campus when used in conjunction with other student and staff data.…

Automated scoring models for the "e-rater"® scoring engine were built and evaluated for the "GRE"® argument and issue-writing tasks. Prompt-specific, generic, and generic with prompt-specific intercept scoring models were built and evaluation statistics such as weighted kappas, Pearson correlations, standardized difference in…

The goal of this article is to contribute to the validation of a self-evaluation method, which can be used by schools to evaluate the quality of their Competence Assessment Program (CAP). The outcomes of the self-evaluations of two schools are systematically compared: a novice school with little experience in competence-based education and…

Using 65 items from a mental health screening questionnaire, the History Opinion Inventory-Revised (HOI-R), the present study compared three strategies of scale construction--(1) internal (based on factor analysis), (2) external (based on empirical performance) and (3) intuitive (based on clinicians' opinion)--to predict whether 203,595 U.S. Air…

This research evaluated the inter-rater reliability of the Performance Assessment for California Teachers (PACT). Multiple methods for estimating overall rater consistency include percent agreement and Cohen's Kappa (1960), which yielded discrepancies between rater agreement in terms of whether candidates passed or failed particular PACT rubrics.…

Contemporary trends in social service delivery systems require human service agencies to engage in greater levels of advocacy to reform structures and protect programs that serve vulnerable populations. Objective: The purpose of this study was to develop an instrument to measure the policy advocacy behavior of nonprofit human service agencies.…

The purpose of this article is to inform readers about the types of instruments available for assessing and improving mentoring in organizations. Extensive review of the psychological, business and medical literature was conducted to identify commercially published, practitioner-oriented instruments. All of the instruments that were…

Schoolwide positive behavioral interventions and supports (SWPBIS) are becoming increasingly popular with schools across the country to help create safer learning environments for students. An important aspect of SWPBIS is the ongoing monitoring and evaluation of implementation fidelity. Although a few measures have been created to assess the…