We present the development and validation of a new assessment tool, the Moon Phases Concept Inventory for Middle School (MPCI-MS), a concept inventory about the phases of the moon targeting students aged 10 to 14 years old. Items in the questionnaire are based on a careful examination of the concept domain of phases of the moon, ideas and concepts necessary to understand the mechanism of lunar phases, as chosen by a panel of seven professional astronomers. Questions and multiple-choice answers were tested for readability with 5th grade students, tested for reading level, and submitted to a second panel of professional astronomers to check for face and construct validity of the items. The MPCI-MS was tested with N = 296 students from grade 5 in elementary school to secondary 2 (M[subscript age] = 10.2 to 14.1). One item about global perspective on lunar phases had to be removed because of poor psychometric properties. The revised MPCI-MS has a post-test Cronbach alpha score of 0.786 and good overall psychometric properties: the mean difficulty index for the MPCI-MS pretest is 0.47, and 0.61 for the post-test; mean point-biserial correlation (post-test) is 0.376. Test-retest without instruction at one-week interval showed high test-retest reliability [M[subscript pre] = 13.696, M[subscript post] = 14.523; t(45) = 1.315, p = 0.192]. We conclude that the MPCI-MS is a reliable and valid instrument that can discriminate between novices and experts, and can be used to assess 10 to 14 year-old students' learning gains on the topic of lunar phases. The final version of MPCI-MS is a 19-item instrument, including two new questions about eclipses, that takes between 15 and 25 min for students to complete.