Plums tend to fruit heavily on a cycle of three to four years and are best when growing vigorously and healthily. Might I suggest you feed yours - a spring mulch with something quite rich (well composted horse manure), which will also help to conserve moisture from winter rains. We've had a few dry springs recently and the stresses on a tree are significant just when it's trying to grow, flower, and set fruit. I wouldn't chop it down yet. Give it a chance.