Episodic memory refers to a complex and multifaceted process which enables the retrieval of richly detailed evocative memories from the past. In contrast, semantic memory is conceptualized as the retrieval of general conceptual knowledge divested of a specific spatiotemporal context. The neural substrates of the episodic and semantic memory systems have been dissociated in healthy individuals during functional imaging studies, and in clinical cohorts, leading to the prevailing view that episodic and semantic memory represent functionally distinct systems subtended by discrete neurobiological substrates. Importantly, however, converging evidence focusing on widespread neural networks now points to significant overlap between those regions essential for retrieval of autobiographical memories, episodic learning, and semantic processing. Here we review recent advances in episodic memory research focusing on neurodegenerative populations which has proved revelatory for our understanding of the complex interplay between episodic and semantic memory. Whereas episodic memory research has traditionally focused on retrieval of autobiographical events from the past, we also include evidence from the recent paradigm shift in which episodic memory is viewed as an adaptive and constructive process which facilitates the imagining of possible events in the future. We examine the available evidence which converges to highlight the pivotal role of semantic memory in providing schemas and meaning whether one is engaged in autobiographical retrieval for the past, or indeed, is endeavoring to construct a plausible scenario of an event in the future. It therefore seems plausible to contend that semantic processing may underlie most, if not all, forms of episodic memory, irrespective of temporal condition.