Coefficient Kappa is generally defined in terms of procedures of computation rather than in terms of a population. Here a population definition is proposed. Factors influencing the magnitude of Kappa are identified. Strategies to improve reliability are proposed, including that of combining multiple unreliable diagnoses. (Author/CTM)

This study explores a strategy for improving the stability of variance component estimates when only small samples are available, using a series of small, less complex generalizability (G) study designs as a surrogate for a single large design. (Author/BW)

Problems of type I errors associated with multiple comparisons in the same experiment are discussed. A table is provided for the rapid determination of experimentwise alpha level when a number of independent statistical tests are employed in the same experiment. Suggested applications and the rationale for this procedure are supplied. (Author/JKS)

Many interactions are found with categorical variables but few with continuous variables, suggesting that, from a mathematical point of view, investigating interactions with continuous variables is less powerful than with categorical variables. This issue is analyzed from three other points of view: study design, measurement of the independent…

Describes differences between multiple correlation analysis (MCA) and multiple regression analysis (MRA), showing how these approaches involve different research questions and study designs, different inferential approaches, different analysis strategies, and different reported information. (SLD)

Evaluation of the binomial effect size display (BESD), proposed as a format for presenting effect sizes associated with certain research, suggests that its application is limited to presenting results of 2 X 2 tables in which the overall binomial success rate is 50%. Problems with BESD use are explored. (SLD)

Notes that, when planning research, obvious strategy is to assess likely flaws and develop approaches to overcome them. Introduces set of concepts that researchers can use to evaluate quality of their designs. Presents 32 threats to validity organized according to 4 types of validity (internal, external, statistical conclusion, and construct…

Three approaches to the analysis of main and interaction effect hypotheses in nonorthogonal designs were compared in a 2 x 2 design for data that was neither normal in form nor equal in variance. The Welch-James test with trimmed means and Winsorized variances provided excellent Type I error control. (SLD)

Presents examples of egregious errors of interpretation in time-series intervention models and makes recommendations regarding the correct specification of the design matrix. Discusses the profound effects of variants of the slope change variable in the design matrix. (SLD)

The individual best qualified to design an empirical study is one who is most familiar with the nature of the phenomenon and research area; most familiar with the possible alternative methods for designing and analyzing the study; and most capable of evaluating the potential advantages and disadvantages of the alternatives. (Author)

This guide is designed to help districts and schools evaluate reform efforts related to the American Recovery and Reinvestment Act (ARRA), including School Improvement Grants. It includes practical, how-to tips for conducting program evaluations, whether one takes a "do-it-yourself" approach or seeks outside assistance to evaluate reform…

This study explored career pathways for becoming an athletic director (AD) at National Collegiate Athletic Association (NCAA) Divisions I, II, and, III member institutions with consideration of gender and race/ethnicity. The study employed an exploratory, descriptive research design using a quantitative electronic survey tapping a census of all…

This qualitative case study endeavored to understand how program managers at one post-secondary Canadian university define, understand and undertake the marketing of their revenue-generating English as a Second Language (ESL) programs. Data were collected through interviews with three managers of distinct English as a Second Language (ESL)…

The article suggests that though narrative inquiry as a research methodology entails free conversations and personal stories, yet it should not be totally free and fictional as it has to conform to some recognized standards used for conducting educational research. Hence, a qualitative study conducted by Russ (1999) was explored as an exemplar…

This paper reports a systematic literature review examining empirical studies on the effects of peer assessment for learning. Peer assessment is fundamentally a social process whose core activity is feedback given to and received from others, aimed at enhancing the performance of each individual group member and/or the group as a whole. This makes…