Small non-coding RNAs (smRNA) participate in the regulation of development, cell differentiation, adaptation to environmental constraints and defense responses in plants. They negatively regulate gene expression by degrading specific mRNA targets, repressing their translation or modifying chromatin conformation through homologous interaction with target loci. MicroRNAs (miRNA) and short-interfering RNAs (siRNA) are generated from long double stranded RNA (dsRNA) that are cleaved into 20–24-nucleotide dsRNAs by RNase III proteins called DICERs (DCL). One strand of the duplex is then loaded onto effective complexes containing different ARGONAUTE (AGO) proteins. In this review, we explored smRNA diversity in model legumes and compiled available data from miRBAse, the miRNA database, and from 22 reports of smRNA deep sequencing or miRNA identification genome-wide in three legumes: Medicago truncatula, soybean (Glycine max) and Lotus japonicus. In addition to conserved miRNAs present in other plant species, 229, 179, and 35 novel miRNA families were identified respectively in these 3 legumes, among which several seems legume-specific. New potential functions of several miRNAs in the legume-specific nodulation process are discussed. Furthermore, a new category of siRNA, the phased siRNAs, which seems to mainly regulate disease-resistance genes, was recently discovered in legumes. Despite that the genome sequence of model legumes are not yet fully completed, further analysis was performed by database mining of gene families and protein characteristics of DCLs and AGOs in these genomes. Although most components of the smRNA pathways are conserved, identifiable homologs of key smRNA players from non-legumes, like AGO10 or DCL4, could not yet be detected in M. truncatula available genomic and expressed sequence (EST) databases. In contrast to Arabidopsis, an important gene diversification was observed in the three legume models (for DCL2, AGO4, AGO2, and AGO10) or specifically in soybean for DCL1 and DCL4. Functional significance of these variant isoforms may reflect peculiarities of smRNA biogenesis and functions in legumes.