ChatGPT has emerged as a popular choice in education that has transformed the experience for both teachers and students. This study investigates the performance of ChatGPT in aiding learning in the biochemistry classroom in two ways. We sought to determine how effective ChatGPT 3.5 was in generating study materials for an introductory biochemistry course. A lecture was delivered in class and transcribed. The resulting transcript was curated, then submitted to ChatGPT 3.5 to generate a summary, a set of notes, and an outline. Each artifact was verified and given to students to help prepare for a quiz. Students were asked about the use of AI-generated materials compared to other study materials. Results were bimodal for the AI-generated materials, with some students indicating that the materials were useful while others preferred traditional study materials such as the text or class notes. We also compared the performance of ChatGPT on open-note biochemistry exams that students had taken. The exams were completed by ChatGPT in two modes: with and without access to external tools. Performance metrics were used to evaluate ChatGPT performance and student responses. The results showed that ChatGPT was unable to pass the exams. This research provides valuable insights into the potential of ChatGPT in educational settings, highlighting strengths and limitations. The implications of these findings may inform the design of future AI-assisted tools and contribute to the ongoing discussions surrounding the integration of AI in education.