You can give it a light trim over to remove most with a pair of shears, but only lightly, or you can just leave them alone, it'll be fine either way.

On a personal level, I found Ivyei to be one of the worst shrubs I have ever planted. The nature of losing lower leaves, yellowing leaves and holding on to the dead heads made it look unsightly so much that I spent ages dead-heading an entire hedge. So as Bamboo suggests, I would certainly give it a light trim, but do it soon, as you will encourage lots of new sappy shoots which will become quite tender if we have early frosts.