Cross-dimensional interference between spatial and temporal processing provides valuable insights into the neuronal representation of space and time. Previous research has frequently found asymmetric interference patterns, with temporal judgments being more affected by spatial information than vice versa. However, this asymmetry has been attributed to the predominant use of visual paradigms (e.g., participants judge the size or duration of visual stimuli), which might facilitate spatial over temporal processing. It has been suggested that the asymmetry vanishes or even reverses when auditory stimuli are used. To test this assumption, we took advantage of the fact that acoustic reverberation carries information about the physical size of rooms. Participants judged either room size or duration, with stimuli being presented either in the visual (rooms presented in virtual reality) or the auditory modality (reverberation-based sounds). For both modalities, we found that judgments about room size were influenced by irrelevant temporal information, while judgments about duration remained unaffected by irrelevant spatial information. As time judgments were consistently rated as more difficult relative to space judgments, this pattern of interference cannot be explained on the basis of task difficulty. These results demonstrate the flexibility of space-time interference and challenge the assumption that the representation of time is necessarily based on spatial representations.