The sensorimotor theory of infancy has been overthrown, but there is little consensus on a replacement. We hypothesize that a capacity for representation is the starting point for infant development, not its culmination. Logical distinctions are drawn between object representation, identity, and permanence. Modern experiments on early object permanence and deferred imitation suggest: (a) even for young infants, representations persist over breaks in sensory contact, (b) numerical identity of objects (Os) is initially specified by spatiotemporal criteria (place and trajectory), (c) featural and functional identity criteria develop, (d) events are analyzed by comparing representations to current perception, and (e) representation operates both prospectively, anticipating future contacts with an O, and retrospectively, reidentifying an O as the “same one again.” A model of the architecture and functioning of the early representational system is proposed. It accounts for young infants’ behavior toward absent people and things in terms of their efforts to determine the identity of objects. Our proposal is developmental without denying innate structure and elevates the power of perception and representation while being cautious about attributing complex concepts to young infants.