In Britain, if there's ants all over a plant, it's because there's blackfly (or aphids) on it - the ants like the honeydew they excrete. Check for aphids, spray with whatever is suitable in your country.

I was about to assume that this was passiflora caerulea the hardy passion flower and that this was due to the cold winter in the UK, and then I noticed you are in India!
My passiflora edulis has gone very yellow in our conservatory as it always does this in winter before recovering in spring.
I'd be interested to know which of the dozens of passion flowers you are growing, as some are very sensitive to temperature. Some are tropical varieties, some grow well in Mediterranean climates, and some like caerulea are hardy.
I doubt very much that the ants or aphids are the problem.