As a vital learning activity in second language (L2) writing classrooms, peer feedback plays a crucial role in improving students' writing skills. However, student reviewers face challenges in providing impactful feedback on peers' essays. Low-quality peer reviews emerge as a persistent problem, adversely affecting the learning effect of peer feedback. To enhance students' peer feedback provision, this study introduces EvaluMate, an AI-supported peer review system, which incorporates a chatbot named Eva, designed to evaluate and provide feedback on student reviewers' comments on peers' essays. Forty-four Chinese undergraduate students engaged with EvaluMate, utilising its features to generate feedback on peers' English argumentative essays. Chat log data capturing the students' interactions with the chatbot were collected, including the comments they wrote on peer essays and the feedback offered by the chatbot on their comments. The results indicate that the integration of AI supervision improved the quality of students' peer reviews. Students employed various strategies during their comment revision in response to AI feedback, such as introducing new points, adding details, and providing illustrative examples, which helped improve their comment quality. These findings shed light on the benefits of AI-supported peer review systems in empowering students to provide more valuable feedback on peers' written work.