While the importance of feedback in education is well established, the effects of structured knowledge in collaborative academic writing remains uncertain. This study introduces an academic writing feedback tool that combines structured knowledge mining, analysis, and visualization. An empirical experiment was conducted in a second-year university class with fifty-five students to examine the impact of the tool on different writing phases. Multiple data sources (i.e., scores, peer comments, discussions, surveys, and interviews) are collected and analyzed using a mixed-method approach. The findings demonstrate that structured knowledge feedback significantly improves specific metrics used to assess academic writing, leading to an overall enhancement in writing quality. The intervention also influences students' engagement, both behaviorally and cognitively, during online discussions and peer comment phases. Moreover, all students exhibited a positive perception of the writing feedback tool and considered peer comments as the most beneficial collaborative phase when structured knowledge intervention was employed. However, their preferences regarding the presentation form of feedback varied. Finally, the study provides implications for the development and research of NLP-powered (Natural Language Processing) feedback tools. These insights aim to inspire future studies on collaborative academic writing, emphasizing the potential of structured knowledge feedback in fostering effective writing practices.