Background: The use of chatbots as learning assistants is receiving increasing attention in language learning due to their ability to converse with students using natural language. Previous reviews mainly focused on only one or two narrow aspects of chatbot use in language learning. This review goes beyond merely reporting the specific types of chatbot employed in past empirical studies and examines the usefulness of chatbots in language learning, including first language learning, second language learning, and foreign language learning. Aims: The primary purpose of this review is to discover the possible technological, pedagogical, and social affordances enabled by chatbots in language learning. Materials & Methods: We conducted a systematic search and identifies 25 empirical studies that examined the use of chatbots in language learning. We used the inductive grounded approach to identify the technological and pedagogical affordances, and the challenges of using chatbots for students' language learning. We used Garrison's social presence framework to analyze the social affordances of using chatbots in language learning. Results: Our findings revealed three technological affordances: timeliness, ease of use, and personalization; and five pedagogical uses: as interlocutors, as simulations, for transmission, as helplines, and for recommendations. Chatbots appeared to encourage students' social presence by affective, open, and coherent communication. Several challenges in using chatbots were identified: technological limitations, the novelty effect, and cognitive load. Discussion and Conclusion: A set of rudimentary design principles for chatbots are proposed for meaningfully implementing educational chatbots in language learning, and detailed suggestions for future research are presented.