Looks like Arum maculatum. (aka Cuckoo pint, Lords and ladies, Jack in the pulpit and others...) The leaves disappear before the seed heads ripen. Or possibly the larger Arum italicum which is taller and has plain flowers rather than spotted ones..

I agree, the birds love the berries but left to grow you will rapidly get lots of plants.

welcome to GoY too :o)