Associations between language and space are of central interest for grounded models of language comprehension. Various studies show that reading words such as "bird" or "shoe" results in faster responses toward the typical location of the corresponding entity (e.g., after "bird", upward responses are faster than downward responses). Critically, as of yet, the mechanisms underlying these effects and their boundary conditions are widely unknown. In fact, it cannot be ruled out that these effects are by-products of processing that only occur in very specific task settings. Here we investigated the role of 3 major processes (response set, labeling, and saliency) that might underlie these compatibility effects in Stroop-like paradigms. In Experiment 1, we aimed at overcoming the binary nature of the response set by including responses both in the vertical and the horizontal dimension. In Experiment 2 no memorizing of the color-to-response mapping was required, making internal response labeling unnecessary. In Experiment 3 this was replicated in a mouse-tracking setup. In all experiments a clear language-space association was observed. Critically, in a final experiment not only the saliency of verticality in the response set but also in the stimulus set was reduced. Here no language-space association was observed. Together these results suggest that language-space associations extend beyond bipolar response settings and that internal response labeling is not a precondition for finding these compatibility effects. However, the vertical dimension needs to be salient either in the stimulus or response set. Implications for models of language comprehension are discussed.