Research suggests that the inspection service in England, Ofsted, has negative impacts on teachers and creates a culture of fear and performativity. Of particular concern is Ofsted's potential to harm schools and educators as they are navigating the ongoing impacts of the pandemic, growing child poverty, and crises relating to teacher retention, SEND, and mental health. This paper draws on data collected as part of a national survey and focus groups to explore how Ofsted's inspections and judgements cause stress in schools with the highest student needs and how these schools are situated in the context of wider societal neglect. We focus on the perspectives of teachers that work in schools labelled 'Special Measures', 'Inadequate', or 'Requires Improvement', and teachers serving large numbers of disadvantaged students. This paper argues that Ofsted marginalises the most vulnerable schools, and that school improvement is incompatible with Ofsted's culture of fear and deficit model of accountability.