Digital resources offer a vast assortment of educational opportunities for students in higher education. From 2018 to 2022, a digital study assistant (DSA), named SIDDATA, was developed at three German universities and consequently field-tested. One of the DSA's features is an AI-driven natural language interface for educational resource recommendation. This paper performs an analysis of the effectiveness of recommendations, by analyzing data generated over the course of two years of DSA usage. We find that although initial user interest is high, only a small percentage of users engage with the recommendation feature. Furthermore, we find that quality of recommendations was perceived as mixed to negative.