During natural vision the entire retina is stimulated. Likewise, during natural tactile behaviors, spatially extensive regions of the somatosensory surface are co-activated. The large spatial extent of naturalistic stimulation means that surround suppression, a phenomenon whose neural mechanisms remain a matter of debate, must arise during natural behavior. To identify common neural motifs that might instantiate surround suppression across modalities, we review models of surround suppression and compare the evidence supporting the competing ideas that surround suppression has either cortical or sub-cortical origins in visual and barrel cortex. In the visual system there is general agreement lateral inhibitory mechanisms contribute to surround suppression, but little direct experimental evidence that intracortical inhibition plays a major role. Two intracellular recording studies of V1, one using naturalistic stimuli (Haider et al., 2010), the other sinusoidal gratings (Ozeki et al., 2009), sought to identify the causes of reduced activity in V1 with increasing stimulus size, a hallmark of surround suppression. The former attributed this effect to increased inhibition, the latter to largely balanced withdrawal of excitation and inhibition. In rodent primary somatosensory barrel cortex, multi-whisker responses are generally weaker than single whisker responses, suggesting multi-whisker stimulation engages similar surround suppressive mechanisms. The origins of suppression in S1 remain elusive: studies have implicated brainstem lateral/internuclear interactions and both thalamic and cortical inhibition. Although the anatomical organization and instantiation of surround suppression in the visual and somatosensory systems differ, we consider the idea that one common function of surround suppression, in both modalities, is to remove the statistical redundancies associated with natural stimuli by increasing the sparseness or selectivity of sensory responses.