This paper compares the use of confidence intervals (CIs) and a sensitivity analysis called the number needed to disturb (NNTD), in the analysis of research findings expressed as 'effect' sizes. Using 1,000 simulations of randomised trials with up to 1,000 cases in each, the paper shows that both approaches are very similar in outcomes, and each…

This brief paper introduces a new approach to assessing the trustworthiness of research comparisons when expressed numerically. The 'number needed to disturb' a research finding would be the number of counterfactual values that can be added to the smallest arm of any comparison before the difference or 'effect' size disappears, minus the number of…

This paper considers the model of school effectiveness (SE) currently dominant in research, policy and practice in England (although the concerns it raises are international). It shows, principally through consideration of initial and propagated error, that SE results cannot be relied upon. By considering the residual difference between the…

When governments and pressure groups attend only to research that suits their political purposes, it makes a mockery of the idea of evidence-informed policy and practice. In this article, the author cites examples of research that picked up weak and misleading evidence for political purposes. The author argues that researchers are tied in to the…

This paper re-considers some of the evidence for low and declining social mobility in Britain, showing that one study based on a re-analysis of cohort figures appears to have had an impact on policy-makers out of all proportion to its scale and rigor. The study claimed to show that the income of parents and children were more closely related for…