In sparse data meta-analyses (with few trials or zero events), conventional methods may distort results. Although better-performing one-stage methods have become available in recent years, their implementation remains limited in practice. This study examines the impact of using conventional methods compared to one-stage models by re-analysing…

Not knowing the population size is a common problem in data-limited contexts. Drawing on work in Sierra Leone, this short take outlines a four-step solution to this problem: (1) estimate the population size using expert interviews; (2) verify estimates using interviews with participants sampled; (3) triangulate using secondary data; and (4)…

Randomized A/B tests allow causal estimation without confounding but are often under-powered. This paper uses a new dataset, including over 250 randomized comparisons conducted in an online learning platform, to illustrate a method combining data from A/B tests with log data from users who were not in the experiment. Inference remains exact and…

The purpose of this study was to compare 12 commonly-used nonoverlap methods with each other and with the results of visual analysis. Data were obtained from 25 studies focused on embedded instruction and schema-based instruction and included a total of 101 graphs. Treatment effect estimates using 12 nonoverlap methods were calculated for each…

Public administration is struggling to contend with a substantial shift in practice fueled by the accelerating adoption of information technology. New skills, competencies and pedagogies are required by the field to help overcome the data-skills gap. As a means to address these deficiencies, we introduce the Data Science Literacy Framework, a…

This applied research methods report is a guide for state and local education agency policymakers and their analysts who are interested in studying teacher mobility and retention. This report provides the foundational information needed to answer policy-relevant research questions related to teacher mobility and retention and presents the decision…

The current study aimed to review studies on computational thinking (CT) indexed in Web of Science (WOS) and ERIC databases. A thorough search in electronic databases revealed 96 studies on computational thinking which were published between 2006 and 2016. Studies were exposed to a quantitative content analysis through using an article control…

Apprentice and trainee data are reported by the State and Territory Training Authorities to National Centre for Vocational Education Research (NCVER) on a quarterly basis, starting at the September quarter of 1994. The set of data submitted that quarter is referred to as Collection 1. The sets of data submitted in subsequent quarters are referred…

This methodological brief introduces basic procedures and issues for conducting a high-quality meta-analysis in gifted education. Specifically, we discuss issues such as how to select a topic and formulate research problems, search for and identify qualified studies, code studies and extract data, choose and calculate effect sizes, analyze data,…

Every year the U.S. Department of Education reports for each state in the country the grade levels, subject areas, and geographic areas that have experienced teacher shortages (U.S. Department of Education, Office of Postsecondary Education, 2015). A teacher shortage occurs when the number of teachers available in a specific grade, subject matter…

In an earlier paper, we presented methods for estimating achievement gaps when test scores are coarsened into a small number of ordered categories, preventing fine-grained distinctions between individual scores. We demonstrated that gaps can nonetheless be estimated with minimal bias across a broad range of simulated and real coarsened data…

Ho and Reardon (2012) present methods for estimating achievement gaps when test scores are coarsened into a small number of ordered categories, preventing fine-grained distinctions between individual scores. They demonstrate that gaps can nonetheless be estimated with minimal bias across a broad range of simulated and real coarsened data…

Racial/ethnic diversity has become an increasingly important variable in the social sciences. Research from multiple disciplines consistently demonstrates the tremendous impact of ethnic diversity on individuals and organizations. Investigators use a variety of measures, and their choices can affect the conclusions that can be drawn and limit the…

Nearly all quantitative analyses in higher education draw from incomplete datasets-a common problem with no universal solution. In the first part of this paper, we explain why missing data matter and outline the advantages and disadvantages of six common methods for handling missing data. Next, we analyze real-world data from 5,905 students across…

To date, evaluation research and policy analysis have focused mainly on average program impacts and paid little systematic attention to their variation. Recently, the growing number of multi-site randomized trials that are being planned and conducted make it increasingly feasible to study "cross-site" variation in impacts. Important…