Trafficking of human papillomaviruses to the Golgi apparatus during virus entry requires retromer, an endosomal coat protein complex that mediates the vesicular transport of cellular transmembrane proteins from the endosome to the Golgi apparatus or the plasma membrane. Here we show that the HPV16 L2 minor capsid protein is a retromer cargo, even though L2 is not a transmembrane protein. We show that direct binding of retromer to a conserved sequence in the carboxy-terminus of L2 is required for exit of L2 from the early endosome and delivery to the trans-Golgi network during virus entry. This binding site is different from known retromer binding motifs and can be replaced by a sorting signal from a cellular retromer cargo. Thus, HPV16 is an unconventional particulate retromer cargo, and retromer binding initiates retrograde transport of viral components from the endosome to the trans-Golgi network during virus entry. We propose that the carboxy-terminal segment of L2 protein protrudes through the endosomal membrane and is accessed by retromer in the cytoplasm.