This article presents the author's address at the 2007 "Journal of Applied Quantitative Methods" ("JAQM") prize awarding festivity. The festivity was included in the opening of the 4th International Conference on Applied Statistics, November 22, 2008, Bucharest, Romania. In the address, the author reflects on three theses that…

Researchers are urged to guard against presumption, and to focus on the issues gradually, as they emerge under observation. Naturalistic evaluation studies represent an interaction between a research viewpoint and a field activity. There is uniqueness to the research team's predilection, and uniqueness to the site. Progressive focusing requires…

While evaluating the effectiveness of a portion of the public information system at the Toledo Art Museum, nonclassical approaches to data-gathering presented problems in measuring program aspects which are not easily quantified. There is a necessity to clarify what kinds of information can validly be gathered with naturalistic techniques for an…

On behalf of the Joint Committee on Standards for Educational Evaluation, the author reviews the Committee's recent efforts and discusses the needs for further work related to the "Standards for Evaluations of Educational Programs, Projects, and Materials." The Committee has developed a program to guide its work. The program addresses…

Educational criticism, based largely on a paradigm of the arts, is an evaluation mode which relies heavily on metaphor, other tropes, and artistic language in general, to convey the quality of educational experiences. Recently educational criticism came under fire by Patton, largely for its use of metaphor which Patton believes the critic brings…

Observational approaches used by ecological psychologists can focus on the behavior of persons or on their settings. The options for subject-centered observation are diary methods, specimen records, time sampling, and event sampling or critical incident recording. Serious limitations correlated to the time interval method, as it is often applied,…

Some of the methodological problems in analyzing and reporting qualitative data are addressed, and some solutions based on recent experiences in conducting several qualitative studies are proposed. The following are some guidelines for implementing an intuitive approach to analyzing and reporting qualitative data: (1) use a team of evaluators; (2)…

Describes special program on live supervision in family therapy given at 1988 annual American Association for Counseling and Development (AACD) Conference that featured experts Barbara Okun and Fred Piercy discussing "cotherapy" and "one-way mirror" live supervision approaches. Provides edited transcript of their program.…

Presidential address at 1990 annual meetings of the American Association of Suicidology provides overview of how suicide assessment could be improved in the next decade. Focuses on new assessment rationale and techniques to improve suicide assessment. Describes both suicide assessment techniques that have been used for years and newer techniques…

Because human beings are different and are necessarily subjective, and because evaluation always involves the human factor, many problems arise. The problems may be compounded in third world settings because they have been studied less, and are less understood, and because evaluators are often from outside the setting and therefore bring…

The most important distinction between evaluation (in the psychometric tradition), ethnography, and auditing is that they are guided by three distinctively separate principles. The underlying principle guiding evaluation is assessment. Ethnography is guided by description. Auditing uses description and assessment to establish an opinion on…

It is assumed that the delineation of values is critical to proper needs assessments. "Need" is defined as a requisite or desideratum generated as a discrepancy between a target state (T) and an actual state (A), if and only if the presence of the conditions defined by T can be shown significantly to harm, indispose, or constrain a subject (S).…

Some of the most important evaluation questions cannot be answered through experimental research. The "assessment" of the St. Louis Leadership Program (SLLP), sponsored by the Danforth Foundations, is used as an illustration of a "connoisseur-based study" approach to evaluation. The project team, given limited time and funds, decided that it could…

Sources of confusion about the distinction between process and product measurement are identified, and the effect of the distinction on evaluation quality is shown. Process measurement involves the measure of behavior, performance of a task which generates no products, an independent variable or treatment, use in diagnosing performance, and means.…

This paper reviews what has been learned about evaluation ultilization during the past 20 years. Evaluation utilization is discussed in terms of what is used, who uses evaluation, when evaluation is used, how evaluation is used, where evaluation is used, and why evaluation is used. It is suggested that the personal factor - the interests and…