The ability to reach a large number of users on social networking sites makes it easy for anyone to be exposed to a phenomenon known as cyberbullying. To address this issue, researchers have proposed different classification models for the automatic detection of cyberbullying using machine learning. Yet, the state of incidents related to cyberbullying remains somehow unresolved. To better understand the cause of the problem, we conduct a systematic literature review of previous studies on cyberbullying detection techniques in text. Our findings indicate that cyberbullying classification is achieved through two types of machine learning techniques, Traditional Machine Learning and Representational Machine Learning, with fairly good performance. However, on closer look and analysis, several gaps in previous studies are noted. Those gaps when taken into consideration raise serious questions regarding the performance scores reported in previous studies, the effectiveness of the classification systems proposed, and their application to the real world. We conclude our work by providing some pointers, which can further guide research in building practical solutions to address the problem of cyberbullying.