Relations between memory and the self are framed from a number of perspectives—developmental aspects, forms of memory, interrelations between memory and the brain, and interactions between the environment and memory. The self is seen as dividable into more rudimentary and more advanced aspects. Special emphasis is laid on memory systems and within them on episodic autobiographical memory which is seen as a pure human form of memory that is dependent on a proper ontogenetic development and shaped by the social environment, including culture. Self and episodic autobiographical memory are seen as interlocked in their development and later manifestation. Aside from content-based aspects of memory, time-based aspects are seen along two lines—the division between short-term and long-term memory and anterograde—future-oriented—and retrograde—past-oriented memory. The state dependency of episodic autobiographical is stressed and implications of it—for example, with respect to the occurrence of false memories and forensic aspects—are outlined. For the brain level, structural networks for encoding, consolidation, storage, and retrieval are discussed both by referring to patient data and to data obtained in normal participants with functional brain imaging methods. It is elaborated why descriptions from patients with functional or dissociative amnesia are particularly apt to demonstrate the facets in which memory, self, and personal temporality are interwoven.