In this article, we advocate for and propose a novel concept map driven knowledge tracing (CMKT) model, which utilizes educational concept map for learner modeling. This article particularly addresses the issue of learner data sparseness caused by the unwillingness to practice and irregular learning behaviors on the learner side. CMKT considers the concept map as a new information source and explicitly exploits its inherent information to help the estimation of the learner's knowledge state. Specifically, the pairwise educational relations in the concept map are formulated as the ordering pairs and are used as mathematical constraints for model construction. The topology information in the concept map is extracted and used as the model input by employing the network embedding techniques. Integrating both educational relation information and topology information in the concept map, CMKT adopts the recurrent neural network to perform knowledge tracing tasks. Comprehensive evaluations conducted on five public educational datasets of four different subjects (more than 8000 learners and their 300,000 records) demonstrate the promise and effectiveness of CMKT: The average area under ROC curve (AUC) and overall prediction accuracy (ACC) achieve 0.82 and 0.75, respectively, and CMKT outperforms all the baselines by at least 12.2% and 9.2% in terms of AUC and ACC.