clover is a plant that grows in poorer soils and it improves the quality of the soil. Rattle will certainly help reduce grass growth but not really sure how to tackle your problem.

I would dig up the clover roots and all, and mix some grit and raw sawdust into the soil, and maybe some lime, if you aren't trying to grow acid lovers. Keep an eye on the bed after you sow your wildflower seeds, since clover seeds can be long lived, and some of the cheaper wildflower mixes actually contain clover of one kind or another.

Usual advice for a flower meadow is to plant on poor soil, and its recommended to remove the top layer if the ground is very fertile. Yellow rattle is a parasite on grass so wouldn't affect the clover.

The only other way of reducing fertility I can think of is to delay the meadow idea and plant a hungry crop on the land for a couple of years. (Blackcurrants would flourish!lol)