The holes you describe are about the right size for ground bees, but they shouldn't be contributing to problems with your tree. Ground bees are solitary and each one lives in its own little hole - you should have seen them moving about if that's the cause of the holes.

Otherwise, re the plant, a photograph might help, but inspect the whole plant from top to bottom, in particular the trunk and the base - you're looking for anything that shouldn't be there such as soft spots, cracks, weeping areas, insect infestation, evidence of fungal infection and so on.

And do add another comment here describing what you find