Cognitive load is a critical internal state associated with learners' learning process and significantly influences learning outcomes. With the worldwide popularity of video-based learning (VBL), tracking real-time cognitive load variations becomes more and more important for the timely provision of adaptive learning support during the learning process. This study proposed and validated an electroencephalogram (EEG)-supported approach to tracking real-time cognitive load variations during continuous VBL. We recruited 108 healthy adult participants to watch a specially designed video lecture with a sequence of interconnected slides of equal length. EEG signals were continuously recorded throughout the session. The video lecture was designed with varying levels of content difficulty (i.e., rated from 1 to 5) across slides and was narrated at three different speeds (i.e., slow, normal and fast) to induce cognitive load variations. For each slide, the cognitive load was quantified using both subjective ratings (i.e., self-reported difficulty) and an EEG-derived measure (i.e., alpha power). Through linear mixed model analysis, we demonstrated the feasibility of using alpha power to track real-time cognitive load variations during the continuous VBL process after controlling the effect of mental fatigue. This study provides a foundation for developing learning enhancement technologies that enable the timely provision of adaptive learning support in VBL.