Previous studies have attributed multiple diverse roles to the posterior superior temporal cortex (STC), both visually driven and cognitive, including part of the default mode network (DMN). Here, we demonstrate a unifying property across this multimodal region. Specifically, the lateral intermediate (LIM) portion of STC showed an unexpected feature: a progressively decreasing fMRI response to increases in visual stimulus size (or number). Such responses are reversed in sign, relative to well-known responses in classic occipital temporal visual cortex. In LIM, this “reversed” size function was present across multiple object categories and retinotopic eccentricities. Moreover, we found a significant interaction between the LIM size function and the distribution of subjects' attention. These findings suggest that LIM serves as a part of the DMN. Further analysis of functional connectivity, plus a meta-analysis of previous fMRI results, suggests that LIM is a heterogeneous area including different subdivisions. Surprisingly, analogous fMRI tests in macaque monkeys did not reveal a clear homolog of LIM. This interspecies discrepancy supports the idea that self-referential thinking and theory of mind are more prominent in humans, compared with monkeys.