Learners of Japanese as a second language (JSL) find it difficult to learn various sentence patterns. To assist JSL learners with their study of Japanese sentence patterns (JSPs), this paper constructs a human-machine collaborative framework that combines artificial intelligence (AI) techniques with the users' active participation for Japanese grammar knowledge discovery (JGKD). Large amounts of human-annotated samples play a crucial role in training JGKD models. However, collecting numerous human-annotated samples is challenging, time-consuming and expensive. To solve this problem, this framework obtained a satisfactory performance in three steps. First, an unsupervised machine learning algorithm based on K-means clustering with adjusted weights of linguistic features for readability control was utilized to select representative samples. Second, an interactive human-in-the-loop system that assists users in annotating samples by incorporating morphological analysis techniques was constructed. Finally, data augmentation techniques were applied to generate more samples to enhance the diversity of the training samples. Extensive experiments were conducted, and the experimental results demonstrated that the proposed methods can be very helpful in selecting representative samples, generating augmented samples, and achieving satisfactory performance of JGKD. Moreover, questionnaire investigations reported that the proposed framework can reduce the annotation workload and facilitate learning JSPs for the JSL learners.