Two experiments served to test the hypothesis that partially masking speech with pink noise (Experiment 1) or speech babble (Experiment 2) induces particularly pronounced metacognitive illusions in judgments about the distracting effects of task-irrelevant speech on cognitive performance. We hypothesized that the experimental manipulations would have opposite effects on judgments of distraction and objective distraction effects. Specifically, masked speech should be perceived as being more difficult to listen to than pure speech, thereby evoking a subjective experience of relative disfluency. If people rely on a (dis)fluency heuristic, masked speech should be predicted to be more distracting than pure speech. However, given that pink noise and speech babble mask the auditory changes in the speech signal that drive auditory distraction, people should objectively be less distracted by masked speech than by pure speech. The findings of both experiments support this hypothesis. Masked speech evoked a subjective experience of relative disfluency and was predicted to be more distracting than pure speech. However, participants were objectively less distracted by masked speech than by pure speech in a serial-recall task. Even after multiple firsthand experiences of having ignored masked and pure speech during the serial-recall task, participants judged masked speech to have been more, but not less, distracting than pure speech. Partially masking speech thus had opposite effects on judgments of distraction and objective distraction effects. These findings provide evidence of particularly pronounced metacognitive illusions and support the hypothesis that people rely on (dis)fluency as a cue for predicting distraction.