You can remove leaves that are yellow and the old stems if you want to - I think they are called spent scopes? but leave on the green/yellow leaves as they are returning nutrients to the plant. But you dont have to remove them as yellow leaves do not mean a plant is unhealthy. I dont remove anything till the spring as I think it protects them over winter.