No, it won't crop, particularly where you are. You might get the odd olive in a good year, but mostly not, specially contained in a pot.
These are reasonably hardy, so suggest you move it somewhere very sheltered for winter, say against a wall which gets any sun there is, out of cold winds, and not in a frost pocket. Wrap the pot in bubblewrap to insulate the roots. If we have a severe winter, though, you will need to have somewhere to remove it to under cover or inside which is cool, but not below 3 or 4 degrees C during the worst of the onslaught.

Thanks Bamboo - I thought it probably wouldn't fruit, but I will follow your advice in winter.

i have a cple of small olives, and they survived our cold spell this winter pretty well, - minus 13C for 3 days- didnt even lose any leaves.
Most of the other olives in the area lost theirs but were not damaged otherwise. I think they are pretty hardy.
The only thing they absolutely hate are dampness around their roots, it will kill them pretty quickly.
If they are more than a cple of years old dont bother even about watering them where you are!
I really like your garden.