There's not much you can do and they will eventually go away. In a way you're lucky because fungi usually prefer an environment that is rich in organic fertilizer so it's a sign that you have good soil for your plants.

If you are absolutely certain that they are edible mushrooms you could always eat them.

As long as they're not honey coloured toadstools growing close to woody plants, nothing to worry about. Myron's right, you can't do anything about them other than to twist off the toadstools and dispose of them - they are only the fruiting bodies of mycelium threads running through the soil throughout the rockery, and should only be visible at this time of year.