Usual cause is shortage of water when bud initiation takes place during the previous summer, so if you had quite a bit of dry weather late summer last year and didn't water enough, the buds don't initiate. The other possibility is the sun hitting the buds first thing in the morning when they're frosted, causing them to fall off. This answer assumes the Camellias are planted in the ground, though the same info still applies. Its just if they're in pots, there may be other reasons as well.

I was going to say that!