Synesthesia is a phenomenon in which stimulation in one sensory modality triggers involuntary experiences typically not associated with that stimulation. Inducing stimuli (inducers) and synesthetic experiences (concurrents) may occur within the same modality (e.g., seeing colors while reading achromatic text) or span across different modalities (e.g., tasting flavors while listening to music). Although there has been considerable progress over the last decade in understanding the cognitive and neural mechanisms of synesthesia, the focus of current neurocognitive models of synesthesia does not encompass many crucial psychophysical characteristics documented in behavioral research. Prominent theories of the neurophysiological basis of synesthesia construe it as a perceptual phenomenon and hence focus primarily on the modality-specific brain regions for perception. Many behavioral studies, however, suggest an essential role for conceptual-level information in synesthesia. For example, there is evidence that synesthetic experience arises subsequent to identification of an inducing stimulus, differs substantially from real perceptual events, can be akin to perceptual memory, and is susceptible to lexical/semantic contexts. These data suggest that neural mechanisms lying beyond the realm of the perceptual cortex (especially the visual system), such as regions subserving conceptual knowledge, may play pivotal roles in the neural architecture of synesthesia. Here we discuss the significance of non-perceptual mechanisms that call for a re-evaluation of the emphasis on synesthesia as a perceptual phenomenon. We also review recent studies which hint that some aspects of synesthesia resemble our general conceptual knowledge for object attributes, at both psychophysical and neural levels. We then present a conceptual-mediation model of synesthesia in which the inducer and concurrent are linked within a conceptual-level representation. This “inducer-to-concurrent” nexus is maintained within a supramodal “hub,” while the subjective (bodily) experience of its resultant concurrent (e.g., a color) may then require activation of “spokes” in the perception-related cortices. This hypothesized “hub-and-spoke” structure would engage a distributed network of cortical regions and may account for the full breadth of this intriguing phenomenon.