Hopefully when you start disturbing the soil they will clear off to look for a new undisturbed site.
I would guess that your patch hasn't been used for a while.

As John has said once you start really working the ground the ants will decamp.

I've had luck repelling ants with chopped up orange peel and spearmint leaves (not stems). Doesn't always work, though. Boiling water poured into the mound will also work.

If you can afford it then "ant nematodes" work really well but they are not cheep, otherwise keep digging the nest if you can, it will work......

Thanks for your feedback, I will keep digging as you say, and work down the theories. I can't see any mounds, the ants are everywhere. Hopefully my hubby will become immune if they insist on staying.

Wellyboots

I was allergic to ants--tough luck for a nurseryman--but I did eventually become de-sensitized. I was afraid that my co-workers were going to start calling me "Rosy", in the meantime! ; )