The availability of cheap Wi-Fi Internet connections has encouraged schools to adopt Web 2.0 platforms for teaching, with the intention of stimulating students' academic achievement and participation in school. Moreover, during the recent explosion of the COVID-19 crisis that forced many countries to close schools (as well as offices and factories), the widespread diffusion of these applications kept school systems going. Despite their widespread use as teaching tools, the effect of adopting Web 2.0 platforms on student performance has never been rigorously tested. We fill this gap in the literature by analyzing the impact of using Twitter as a teaching tool on high school students' literature skills. Based on a large-scale, randomized controlled trial that involved seventy schools and about 1,500 students, we find that using Twitter to teach literature has an overall negative effect on students' average achievement, reducing standardized test scores by about 25 percent of a standard deviation. The negative effect is stronger on students who usually perform better.