Could be Plum Sawfly or Plum Moth - check the fruits which have fallen by cutting them open to see what's inside - if there's a creamy white larvae with a brown head, its sawfly. In Plum Moth, the fruits are less likely to fall from the tree, but do have entry holes with a caterpillar inside which is pinkish white with a brown head. These caterpillars concentrate their activities around the stone. If its sawfly, the tree will need spraying 7 to 10 days after petal fall with a pesticide spray containing either Dectamethrin, or Lambda Cyhalothorin. This treatment also works for Plum Moth, but there's an easier way for that - get a pheremone trap and erect it in the tree, following the instructions on the pack.

Thanks for your help

do they all fall off - if not it could just be natural shedding, usually called June drop I think.

I have about 6 plums left

That's better than our new tree!!! We had a new Victoria a couple of years ago. This year there was a lot of blossom but ONE PLUM only has survived!! I have put a net bag around it to try & save it from the birds!! It would be nice to, at least have a taste!! As it is the only one growing, I wonder if it will be record size?!!! :)