Physical education (PE) research has largely been preoccupied with mainstream (regular) schools. This article reports on part of a larger research project that centralises special school PE. In particular, Gramsci's conceptualisations of hegemony, power and ideology are utilised to help shed light on the key factors that shape the culture of special school PE. A number of key themes were constructed from twelve interviews with special school senior leaders and PE teachers including, 'economic climate: budgetary constraints', 'access to appropriate facilities and learning spaces' and 'pressures from Office for Standards in Education, Children's Services and Skills (Ofsted) and senior management team'. These findings demonstrate how particular historical and contemporary factors contribute to the positioning of PE in special schools. The status and value of PE in these settings is sometimes considered less favourably than other areas of the curriculum or indeed mainstream PE. In spite of this, staff tasked with delivering special school PE had the desire and creativity to offer engaging experiences. In concluding we note that issues concerning economic constraints, limited space to deliver PE and pressures associated with Ofsted can be found in many mainstream schools too. However, honing in on the particular circumstances within special schools broadens insight about PE in contemporary schools.