Evidence for an approximate analog system of numbers has been provided by the finding that the comparison of two numerals takes longer and is more error-prone if the semantic distance between the numbers becomes smaller (so-called numerical distance effect). Recent embodied theories suggest that analog number representations are based on previous sensory experiences and constitute therefore a common magnitude metric shared by multiple domains. Here we demonstrate the existence of a cross-modal semantic distance effect between symbolic and tactile numerosities. Participants received tactile stimulations of different amounts of fingers while reading Arabic digits and indicated verbally whether the amount of stimulated fingers was different from the simultaneously presented digit or not. The larger the semantic distance was between the two numerosities, the faster and more accurate participants made their judgments. This cross-modal numerosity distance effect suggests a direct connection between tactile sensations and the concept of numerical magnitude. A second experiment replicated the interaction between symbolic and tactile numerosities and showed that this effect is not modulated by the participants’ finger counting habits. Taken together, our data provide novel evidence for a shared metric for symbolic and tactile numerosities as an instance of an embodied representation of numbers.