Arabidopsis MOM1 is required for the heritable maintenance of transcriptional gene silencing (TGS). Unlike many other silencing factors, depletion of MOM1 evokes transcription at selected loci without major changes in DNA methylation or histone modification. These loci retain unusual, bivalent chromatin properties, intermediate to both euchromatin and heterochromatin. The structure of MOM1 previously suggested an integral nuclear membrane protein with chromatin-remodeling and actin-binding activities. Unexpected results presented here challenge these presumed MOM1 activities and demonstrate that less than 13% of MOM1 sequence is necessary and sufficient for TGS maintenance. This active sequence encompasses a novel Conserved MOM1 Motif 2 (CMM2). The high conservation suggests that CMM2 has been the subject of strong evolutionary pressure. The replacement of Arabidopsis CMM2 by a poplar motif reveals its functional conservation. Interspecies comparison suggests that MOM1 proteins emerged at the origin of vascular plants through neo-functionalization of the ubiquitous eukaryotic CHD3 chromatin remodeling factors. Interestingly, despite the divergent evolution of CHD3 and MOM1, we observed functional cooperation in epigenetic control involving unrelated protein motifs and thus probably diverse mechanisms.