The use of inter-rater reliability (IRR) methods may provide an opportunity to improve the transparency and consistency of qualitative case study data analysis in terms of the rigor of how codes and constructs have been developed from the raw data. Few articles on qualitative research methods in the literature conduct IRR assessments or neglect to…

The reason for conducting the present case study was the problems the researchers encountered during data collection for another research project (Primary Study) entitled "The effects of virtual versus traditional flipped classes on EFL learners' grammar knowledge, self-regulation, and autonomy." Two online questionnaires were…

In science in general and in the context of single-case experimental designs, replication of the effects of the intervention within and/or across participants or experiments is crucial for establishing causality and for assessing the generality of the intervention effect. Specific developments and proposals for assessing whether an effect has been…

The work presented in this case study results from a study conducted in 2012-2014 examining a newly created teacher evaluation system to determine the inter-rater reliability of the classroom observation instrument. The teacher evaluation system was the result of a partnership between the school district and the university in the same city…

This case study exemplifies the design of a formative evaluation project concerned with the implementation of a new school form in Austria. The purpose of this case study is to show how multiple methods were implemented in the research project NOESIS ("Niederösterreichische Schule in der Schulentwicklung"; Lower Austrian Middle School in…

The use of single-case effect sizes (SCESs) has increased in the intervention literature. Meta-analyses based on single-case data have also increased in popularity. However, few researchers who have adopted these metrics have provided an adequate rationale for their selection. We review several important statistical assumptions that should be…

In January 2014, I began working as an evaluator on an education reform project being implemented by faculty in science and engineering departments at my university. The core goals of the project were for participating faculty members to improve students' learning, engagement, and recruitment and retention in selected introductory-level courses…

This case study describes the use of a regression discontinuity design and a quasi-experimental design to evaluate the impact of School Improvement Grants on high school students' reading, mathematics, and science achievement. The high schools that participated in the School Improvement Grant program were very low performing and received…

We present a case study of the process through which a methodology was developed and applied to a quasi-experimental research study that employed propensity score matching. Methodological decisions are discussed and summarized, including an explanation of the approaches selected for each step in the study as well as rationales for these…

Teaching is a complex construct, and there is near-universal consensus, in the literature in the United States and internationally, that capturing this complexity requires collecting multiple measures from multiple sources. Yet, what measures to use and how is a contentious area of education policy research. In practice, there is little specific…

Reporting standards and critical appraisal tools serve as beacons for researchers, reviewers, and research consumers. Parallel to existing guidelines for researchers to report and evaluate group-comparison studies, single-case experimental (SCE) researchers are in need of guidelines for reporting and evaluating SCE studies. A systematic search was…

Researchers who are interested in asking a number of questions about a particular phenomenon are increasingly looking to mixed methods as a useful research approach. In this case study, we discuss how mixed-methods research is situated in current debates about quantitative and qualitative methods. Then, we provide some reasons as to why we, and…

Evaluation in higher education is an evolving social practice; that is, it involves what people, institutions and broader systems do and say, how they do and say it, what they value, the effects of these practices and values, and how meanings are ascribed. The textual products (verbal, written, visual and gestural) that inform and are produced by,…

This case report describes the methods used to investigate a study concerning the Literacy and Numeracy assessment practices of primary teachers and the impact of educational reform (Blueprint 1). Qualitative case study methods framed by an ethnological approach were useful in the study, and the participants were bounded by the context of the…

Single-case designs are a class of research methods for evaluating treatment effects by measuring outcomes repeatedly over time while systematically introducing different condition (e.g., treatment and control) to the same individual. The designs are used across fields such as behavior analysis, clinical psychology, special education, and…