According to the modulation hypothesis, arousal is the crucial factor in the emotional enhancement of memory (EEM). However, the multifactor theory of the EEM recently proposed that cognitive characteristics of emotional stimuli, e.g., relatedness and distinctiveness, also play an important role. The current study aimed to investigate the individual contribution of arousal to the neural correlates of the EEM by controlling for these additional cognitive factors. We observed the characteristic neuronal correlates of the EEM, in particular enhanced activity in the amygdala and hippocampus, which provides evidence for an arousal-driven EEM in the amygdala as proposed by the modulation hypothesis.