It has long been suspected that touch plays a fundamental role in the calibration of visual perception, and much recent evidence supports this idea. However, as the haptic exploration workspace is limited by the kinematics of the body, the contribution of haptic information to the calibration process should occur only within the region of the haptic workspace reachable by a limb (peripersonal space). To test this hypothesis we evaluated visual size perception and showed that it is indeed more accurate inside the peripersonal space. We then show that allowing subjects to touch the (unseen) stimulus after observation restores accurate size perception; the accuracy persists for some time, implying that calibration has occurred. Finally, we show that observing an actor grasp the object also produces accurate (and lasting) size perception, suggesting that the calibration can also occur indirectly by observing goal-directed actions, implicating the involvement of the “mirror system”.