Training and development programs are increasingly delivered online with numerous studies reporting no differences in learning outcomes between online and traditional learning. However, there are no established standardized methods to evaluate the effectiveness of online learning. This review aims to map the state of research around health-related education to determine what elearning evaluation methods are being used, the strengths or deficiencies of these methods, and which are appropriate for measuring the effectiveness of online education. Databases searched were PubMed, ProQuest, Education Resources Information Centre, Cumulative Index to Nursing and Allied Health Literature, Scopus, PsychInfo, and Medline. Studies were included if they were published between 2011 and 2021, reported health-related online education and included an evaluation component. Thirty studies were obtained from numerous countries with varied methodologies and designs. Participants ranged from undergraduate students to medical professionals. Evaluation methods included student participation, students' reaction to the training program, self-efficacy, knowledge assessment, long-term performance, and the Kirkpatrick Evaluation Framework. The review identified that course evaluations, such as measuring student satisfaction scores alone, are insufficient when used to quantify learning effectiveness for online education. This was particularly important as studies are reporting these single metrics as positive effects of training interventions without justification. Suggestions within the reviewed papers were to adopt and implement an appropriate validated method within the course curriculum to evaluate learning outcomes.