Have you noticed any flowers during the 4 years? There are a few reasons why it has not produced fruit. Firstly, early season frost burns off the flowers, poor pollination due to the fact that there are no other trees of the plum family near by or it just won't flower. I give it another chance and give it a high potash feed now and at the end of September. If next year, it still doesn't produce fruit, then I would cut it down and perhaps try another variety in a south or west facing aspect. Perhaps something like Jubilee which is a modern version of Victoria.