Its Brown Rot - much more common in wet years when the tree is perhaps overloaded with fruit and the fruits are touching one another, but the commonest cause is prior damage allowing the rot to set in. Damage can be caused by Plum Moth, bird pecks or wasps on ripe fruit. If its Plum Moth, then you'll need to get a pheremone trap for next year and possibly spray in June.

Regarding the potato foliage, it would be much better to put it on the compost heap.