The use of measures of magnitude of effect has been advocated as a way to go beyond statistical tests of significance and to identify effects of a practical size. They have been used in meta-analysis to combine results of different studies. Describes problems associated with measures of magnitude of effect (particularly study size) and…

In the social sciences, evaluating the effectiveness of a program or intervention often leads researchers to draw causal inferences from observational research designs. Bias in estimated causal effects becomes an obvious problem in such settings. This paper presents the Heckman Model as an approach sometimes applied to observational data for the…

Several recent studies have investigated the application of statistical inference procedures to the analysis of differential item functioning (DIF) in test items that are scored on an ordinal scale. Mantel's extension of the Mantel-Haenszel test is a possible hypothesis-testing method for this purpose. The development of descriptive statistics for…

Discussion of evaluation methodology in information retrieval focuses on the average precision over a set of fixed recall values in an effort to evaluate the retrieval effectiveness of a search algorithm. Highlights include a review of traditional evaluation methodology with examples; and a statistical inference methodology called bootstrap.…

Results from two Monte Carlo studies in item response theory (comparisons of computer item analysis programs and Bayes estimation procedures) are analyzed with inferential methods to illustrate the procedures' strengths. It is recommended that researchers in item response theory use both descriptive and inferential methods to analyze Monte Carlo…

Studied the impact of changes in item difficulty on National Assessment of Educational Progress (NAEP) estimates over time through a Monte Carlo study that simulated the responses of 1990 NAEP mathematics respondents to 1990 and 1996 NAEP mathematics items. Results support the idea that these changes have not affected the NAEP trend line.…

Researchers, journal editors, textbook authors, and those responsible for writing publication manuals must work together to enhance the thoughtful reporting of statistical results and to make clear the necessity for reporting effect sizes. (SLD)

Suggests that researchers should report the results of appropriate significance tests and the effect sizes associated with each test. Discusses the roles of textbook authors, publication manuals, and journal editors in leading the movement to better statistical reporting. (SLD)

Documents the recent use of one-tailed F-tests in communication journals, and examines the arguments both for and against their use. Examines the use of these tests within the broader unresolved controversy surrounding the use of one-tailed tests. Recommends that future researchers should most often avoid one-tailed Fs, and generally exercise…

Since the 1993 APA task force of the Society of Clinical Psychology developed guidelines to apply data-based psychology to the identification of effective psychotherapy, there has been an increasing number of texts focussing on Empirically based Psychotherapy and Empirically Supported Treatments. This manuscript examines recent key texts and…

Despite the increasing criticism of statistical significance testing by researchers, particularly in the publication of the 1994 American Psychological Association's style manual, statistical significance test results are still popular in journal articles. For this reason, it remains important to understand the logic of inferential statistics. A…

Sixty-seven educational research journals were investigated to determine the frequency of usage of inferential statistical techniques therein. The most frequently used statistical methodologies in the literature reviewed, which utilized inferential approaches, are the following: analysis of variance, correlation, t-test, multiple analysis of…

An on-going study is examining a method proposed by S. Goetz and D. Debertin for estimating demand for educational attainment at the county level by using secondary data. The method produces compact information showing the relationship between employment, occupation, and educational attainment, information not available at the county level. The…

This collection of activities is designed to show how graphing calculators can be used to explore statistics. The activities address such topics as data representation, distributions, and statistical tests. Teaching notes and calculator instructions are included as are blackline masters. (MM)

Describes a technique for statistically comparing bibliometric models and illustrates its use with two examples using Lotka's hypothesis of author productivity and one example using library circulation frequencies. Topics discussed include nested statistical models, analysis of variance, regression, log-linear models, and the likelihood ratio…