This study tested the effects of implementing a narrative computer-based educational game within a middle-school math class. Gameplay consisted of navigating through a virtual spaceship and completing missions by periodically engaging in learning-by-teaching activities that involved helping an avatar solve math problems. In a pretest/posttest matched-groups design, 58 middle-school students either played the game for 10 hours over 4 days in place of their typical math instruction (game group), or they received conventional math instruction that consisted of a matched set of practice problems (control group). Contrary to our hypotheses, results from posttest measures indicated no significant differences in learning outcomes or motivation between the two groups. Importantly, supplementary observational data indicated that students in the game group spent much of their time during gameplay engaging in activities unrelated to the educational content of the game (e.g., navigating the virtual world) and only 20% of their time engaging in learning-by-teaching activities. These results highlight the importance of designing educational games that effectively balance features intended to entertain learners and features intended to promote learning. Implications for implementing educational games into classroom instruction are discussed.