Without seeing a photo of the damage I will guess a weevil of some sort. Go out at night with a flashlight to see what's munching it if you don't see the pest during the day.

Vine weevils eat holes in a couple of our rhododendrons, although unsightly the holes do not affect the rhodos as they are mature specimens. If yours are young then see if you can remove, but it is a thankless job!