The use of concept mapping in research and evaluation has expanded dramatically over the past 20 years. Researchers in academic, organizational, and community-based settings have applied concept mapping successfully without the benefit of systematic analyses across studies to identify the features of a methodologically sound study. Quantitative…

Program theory evaluation, which has grown in use over the past 10 years, assesses whether a program is designed in such a way that it can achieve its intended outcomes. This article describes a particular type of program theory evaluation--logic analysis--that allows us to test the plausibility of a program's theory using scientific knowledge.…

This paper presents a mixed-method approach to measuring the functioning of Safe Schools/Healthy Students (SS/HS) Initiative partnerships. The SS/HS national evaluation team developed a survey to collect partners' perceptions of functioning within SS/HS partnerships. Average partnership functioning scores were used to rank each site from lowest to…

Illicit drug use remains at high levels in the U.S. The federal Office of National Drug Control Policy evaluates the outcomes of national drug demand reduction policies by assessing annual changes in drug use from several federally sponsored annual national surveys. Such survey methods, relying exclusively on drug use as self-reported on…

A theory-driven approach to validity is proposed. The central argument is that a model or theory should be formulated in a program evaluation and the modeling process should include the identification of potential threats to validity in research. (Author/LMO)

Agrees with the power of randomized clinical trials, but makes the case that random assignment raises other, unanticipated threats to internal validity as a result of failing to consider treatment preference in research participant behavior. Reviews benefits and limitations of research strategies that augment randomization while respecting…

The Combined Modified Design is presented to overcome problems of internal and external validity in the conduct of true experiments in longitudinal studies. It combines a modified version of the Randomized Invitation Design with the lottery longitudinal experimental field trial situation. (SLD)

Results of empirical validation studies of six statistical models available for indirectly estimating the need for alcohol, drug abuse, and mental health services across a large geographic area are presented. Models were evaluated using Colorado Social Health Survey data for 4,745 adults from 48 subareas of Colorado. (SLD)

A discussion of current issues in evaluation, related to society's expectation that evaluation is based on scientific authority, is followed by an exploration of concepts of "validity" and "values." The role of the evaluator is increasingly involved in issues of ethics and social justice. (SLD)

Reports on the use of the tailored design method (TDM) to achieve high survey response in two separate studies of the dissemination of Treatment Improvement Protocols (TIPs). Findings from these two studies identify six factors may have influenced nonresponse, and show that use of TDM does not, in itself, guarantee a high response rate. (SLD)

A fuzzy set model is offered for interpreting vague frequency expressions, such as "rarely" and "sometimes." Results with 152 undergraduates reporting marijuana use reflect different frequency expressions for the same level of use and suggest that self-report validity may be enhanced by analyzing frequency expressions as fuzzy…

A model is presented that estimates the rates, numbers, and sociodemographic characteristics of potential clientele for substance abuse programs. Using the synthetic estimates technique and existing bodies of information, the model has been tested and cross validated with substance abuse related problem indicators with good results. (Author/CTM)

A pilot study was designed to investigate two needs assessment approaches: social indicators analysis and key informants surveys. Results showed that social indicators analysis was a valid method of assessing human service needs at a general level. (MH)

A strategy proposed by T. Cook and D. Campbell (1979) was used to investigate attrition in a community-based literacy program, focusing on 287 Head Start families. Data revealed interactions among intervention program-centered characteristics and person-centered characteristics that have implications for study validity and generalizability.…

Factors involved in pre-planned and post-planned evaluation of program effectiveness are compared: (1) reliability and cost of data; (2) internal and external validity; (3) obtrusiveness and threat; (4) goal displacement and program direction. A model to help program administrators decide which approach is more appropriate is presented. (Author/MH)