The popularity of generative AI chatbots, such as ChatGPT, has sparked numerous studies investigating their use in educational contexts. However, it is important to note that chatbots are not a new phenomenon; researchers have explored conversational agents across diverse fields for decades. Conversational agents engage users in natural language conversations through text or voice interfaces. While these agents have demonstrated potential for enhancing human learning, relatively few studies have assessed their overall effectiveness or the contexts in which they are implemented in education. To address this gap, we systematically reviewed empirical studies published before the emergence of ChatGPT. Given the transformative impact of generative AI technologies, we argue that it is crucial to summarize research on conversational agents conducted prior to this paradigm shift. Understanding the educational applications of earlier chatbots provides valuable context for evaluating and guiding ongoing developments in the era of generative AI. Our review examined 3,045 articles, ultimately selecting 23 studies encompassing 29 implementations published between 2004 and 2019. The findings highlight variations in chatbot interfaces, learning modes, and interactions, with evidence of medium to large effects on learning outcomes and positive usability perceptions. Common limitations included non-random sampling methods and small sample sizes. Future research directions emphasize the importance of addressing contextual, implementation, and methodological considerations to advance the field further.