any chance of a photo? the orange flower might be crocosmia and yes they can be removed now as you will be able to follow the leaf down to the corm. remove the corm too.
as for the hydrangea I'd not prune it back until spring. the old foliage help protects the new buds from the worst of the winter. certainly dead head the spent flowers if you want a tidier look.
and welcoe to goy.

Don't cut back the hydrangea til April Rcmp. The faded flower heads protect the buds that will be next year's flowers from frost. When you do cut them back next April cut off the old flower heads back to a nice pair of fat buds (not too far back) and cut out any dead stems back to their base. If it's a mature bush you can thin out some of the oldest stems if the bush is getting crowded.

It's hard to identify a plant without a photograph but the orange plant sounds as if it might be Montbretia. Google to see if it matches what you have.

If you find it is montbretia, it's notorious for becoming overcrowded, and indeed, crowding all other plants out. It's indestructible, so pull some out whenever you like - after it finishes flowering seems like a good time - or when it's coming into growth again next year. You will need to think about how you dispose of the stuff you pull up - every little shred of root, every tiny corm and every seed from its seedheads will produce more plants. Don't put it in a composter. I'd incinerate the bits you want to dispose of, assuming you're allowed to have a bonfire where you live.