Apply systemic herbicide, Round up is popular here in the US.

Assuming you don't want to put glyphosate on them (and this would certainly be easiest and quickest) you could cover them with tarp or old carpet for about six months.Or you could try strimming them hard regularly too -that would discourage them and also prevent them seeding or sending out young ones.They can't survive indefinitely without any leaves in the growing season. I haven't found them regrowing after digging as long as you get the growing points out but they do seed around somewhat.. Maybe it would help to soak the ground first before digging which would enable you to get the fork in even though it will be hard work to lever the plant out. And be vigilant - if you see one reappearing get it out quick before it gets any bigger. Not a lot of time left before they die back for the winter - good luck!
I made the same mistake when starting my present garden so I know where you're coming from!

It sounds as though your soil would benefit from lots of well rotted compost too before you replant.. Use whatever you can get - lawn mowings are a lot better than nothing. They can be left over winter in a deep layer and dug in in the spring. Throw nothing leafy away but compost it.(and what about the geraniums you remove - compost those as well - if the heap gets hot enough it should kill off any seeds).
When you repot containers (if you do) empty the old stuff onto the bed.(not that bed, lol!) Even burying old newspapers would help a bit and you might add some coarse grit to open the soil up an let some air in.

yes they do take over and in the right place they make a lovely carpet of ground cover.

But as you have found they can get out of control. I have resorted to glyphosphate roundup in the past.

If yu go for glyphosate remember that is the chief ingredient in Roundup and its much cheaper to buy it under that name than the trade name..