The first published Fermi large area telescope (Fermi-LAT) measurement of the isotropic diffuse gamma-ray emission is in good agreement with a single power law, and is not showing any signature of a dominant contribution from dark matter sources in the energy range from 20 to 100 GeV. We use the absolute size and spectral shape of this measured flux to derive cross section limits on three types of generic dark matter candidates: annihilating into quarks, charged leptons and monochromatic photons. Predicted gamma-ray fluxes from annihilating dark matter are strongly affected by the underlying distribution of dark matter, and by using different available results of matter structure formation we assess these uncertainties. We also quantify how the dark matter constraints depend on the assumed conventional backgrounds and on the Universe's transparency to high-energy gamma-rays. In reasonable background and dark matter structure scenarios (but not in all scenarios we consider) it is possible to exclude models proposed to explain the excess of electrons and positrons measured by the Fermi-LAT and PAMELA experiments. Derived limits also start to probe cross sections expected from thermally produced relics (e.g. in minimal supersymmetry models) annihilating predominantly into quarks. For the monochromatic gamma-ray signature, the current measurement constrains only dark matter scenarios with very strong signals.