Difficult! You are allowed to cut any ivy back that strays over into your space, but even just the weight on the other side could bring the fence down. Have you tried having a gentle word with him/her yet? My Dad once had an acrimonious altercation with his neighbour about weed seeds, and the feud lasted 15 years, so keep it low key! Our neighbour here had asked the previous owner of this house to cut back a huge ivy plant that was growing up the party stone wall, and shortly after we moved in, came in to our garden uninvited and cut the whole back to within an inch of it's life. Another neighbour snitched on him and I had to go and have strong words. We have since cut the remaining half back ourselves to do maintenance on the wall, and without the ivy to keep it in place, the entire wall has started to crumble and tilt towards our garden. However, stone walls and fences are entirely different things - sorry for rambling on - I think you need to say something. Politely.

i have ivy growing up both sides of our house we are going to kill it off this year hope our walls dont crumble! we are going to cut it at the roots and let it die for a while before we attempt to take it off the walls,would that be the right thing to do?

The only thing you're legally entitled to do, as Gattina suggests, is to cut back the ivy right to their side of the fence. Technically, the cuttings belong to your neighbour, so you're supposed to ask them if they want them - or hand them over. If you can establish that the fence is actually yours and not theirs, then that's slightly different - the law states that, if you want to grow climbers up a fence, you have to have your own support for them, and not use a neighbour's fence as support.
Gerardine - you will need to poison the woody roots at the bottom once you've severed the top growth - use SBK, applied to every single piece of root you can find, large or small, and make holes or cuts in the roots first, where you're going to apply it. That probably means digging around a bit to expose all the roots. As for pulling off the topgrowth - it doesn't loosen its grip even after it's dead, unfortunately, so you run the risk of pulling off any wall covering (stucco, pebbledash, whatever) with it.

Thanks for suggestions, not going to be easy.....

Gerardine, yes that's right. If you leave the stems to dry they usually lift off quite easily, but they do look unsightly in the meantime.If you have a fire you will find the dried stems make wonderful fire lighters!