No, not unless they're nesting in the pot (if the plant is in a pot) when they might, just might chew through roots to enlarge the nest. However, if ants are on your plants, they're there for a reason - check for aphid or scale infestation and if you find it, treat for that - the ants will then disappear.

I lost a rosemary once when they nested underneath it and it was in the ground. If you can't find the reason I'd repot if you aren't squeamish or put some bait down.

If they are nesting in the pot then yes they can kill plants. The amount of acid they produce (Formic acid) can and does kill plants.

Ants won't eat your plants as such as they can't digest them. They will however nibble away at the leaves and carry bits away to build there nest, so a large colony of ants could in theory kill your plants by decimating most of the foliage.

The most likely reason that that your plants are overrun with ants is because they are feeding on the sugary residue left by aphids. Ants love this sticky honey-like nectar as it's a good source of food for them.

Look at the leaves, especially underneath. If you see any insect infestation then spray the leaves with a soap solution such as diluted washing-up liquid. This will get rid of the aphids, the food source for the ants won't be available and so they will move on.