The use of inter-rater reliability (IRR) methods may provide an opportunity to improve the transparency and consistency of qualitative case study data analysis in terms of the rigor of how codes and constructs have been developed from the raw data. Few articles on qualitative research methods in the literature conduct IRR assessments or neglect to…

This paper argues for three aspects of tolerance with respect to QCA research: tolerance with respect to different approaches to QCA; producing QCA research with tolerance (work that is resistant to criticism); and for QCA researchers to be clear about the tolerance of the solutions they present -- especially in terms of calibration and truth…

Despite the rising popularity of big data, there is speculation that evaluators have been slow adopters of these new statistical approaches. Several possible reasons have been offered for why this is the case: ethical concerns, institutional capacity, and evaluator capacity and values. In this method note, we address one of these barriers and aim…

Several programs of research have sought to assess the replicability of scientific findings in different fields, including economics and psychology. These programs attempt to replicate several findings and use the results to say something about large-scale patterns of replicability in a field. However, little work has been done to understand the…

Identifying effective literacy interventions for teachers of very young children is important, but it is not easy. In this case study, we share our experiences implementing a quasi-experimental evaluation of an early literacy program. We describe the design and context of the study and discuss three key issues that we encountered with data…

In 2011, Donna DiBartolomeo and Zachary Clark enrolled in the Arts in Education Program at Harvard Graduate School of Education. Harvard Graduate School of Education is home to Project Zero, an educational research group comprising multiple, independently funded projects examining creativity, ethics, understanding, and other aspects of learning…

"Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research" offers a truly balanced, inclusive, and integrated overview of the processes involved in educational research. This text first examines the general steps in the research process and then details the procedures for conducting specific types…

Confidence intervals (CIs) and effect sizes are essential to encourage meta-analytic thinking and to accumulate research findings. CIs provide a range of plausible values for population parameters with a degree of confidence that the parameter is in that particular interval. CIs also give information about how precise the estimates are. Comparison…

In the May 2010 issue of "Psychological Bulletin," R. E. McGrath, M. Mitchell, B. H. Kim, and L. Hough published an article entitled "Evidence for Response Bias as a Source of Error Variance in Applied Assessment" (pp. 450-470). They argued that response bias indicators used in a variety of settings typically have insufficient data to support such…

Purpose: The purpose of this paper is to critically analyze the value of the written comments section on student evaluations of teaching and develop a framework to improve the interpretability of such data. Design/methodology/approach: The paper reviews past investigations into the reliability and interpretability of student evaluations of…

The application of educational research to practice remains an issue of concern, and yet there has been relatively little consideration of this in relation to reviews of research. While the professional user review hitherto represents the most relevant approach, this involves users applying the findings of an earlier review rather than carrying…

In this article, the authors clarify and provide additional tests of the key elements of their age-period-cohort analysis of non-marital birth rates in this March 2009 issue of "Social Forces." Where Steve Martin, in his commentary, has suggested specific alternative specifications or interpretations of their findings, the authors have…

Meehl's taxometric method was developed to distinguish categorical and continuous constructs. However, taxometric output can be difficult to interpret because expected results for realistic data conditions and differing procedural implementations have not been derived analytically or studied through rigorous simulations. By applying bootstrap…

This paper explores the issues surrounding the study of higher education, usually involving the analysis of structural-functionalism, to less tradition-bound areas representing idealism. It examines these issues through the theoretical framework of symbolic interactionism which, it is noted, rests in the interpretive paradigm. The paper begins…

The purpose of this research was to develop, test, and demonstrate a systematic methodology of triangulation. Triangulation is a technique used to establish credibility of data gathered in qualitative ways. Triangulated conclusions are more stable than any of the individual vantage points from which they were triangulated. Using a previous study…