Here in the States, we call it Bishop's Weed, while in the UK it's called Ground Elder. Sadly, as long as your neighbor has it, it will continue to creep into your garden. All that you can do is keep digging until it is gone from your side--possibly a three or four year project. That's assuming that it doesn't invade the close root area of trees or shrubs, in which case it may never happen!
If you are successful, don't plant anything within 2 feet of the boundary with your neighbor, and inspect that boundary at least once a week as long as the soil isn't frozen, to catch invaders while they are still small. When you find them, shield your plants with a big piece of cardboard, and spray them with an RTU of glyphosphate (not RoundUp, but just glyphosphate).

yes it is in deed ground elder. the variegated form is supposedly less invasive than the native all green one.
I agree with tug on how to keep it under control.

I did finally remove it from a large border but I did double dig and sieve the soil to get all the 'pips' [small storage organ] out. I remember it was feb/march 1996. I didn't plant anything for 6 months and any I saw was immediately removed.

Thank you for the great information on this bothersome plant!

Be careful with the glyco. If you spray the leaves this is going to be transported to the roots which are in your neighbor's property. This will kill the groundcover on your neighbors side too. Not a pleasant prospect.

Thank you, good to know. Really like my neighbors so would not want that to happen!

In my experience with other species, a sprig or two on the other side might die, but most of it should be unaffected. The trouble is that I have no direct experience with ground elder, and so don't know whether glyphosphate wouldn't spread farther in it.