No, don't cut them down. Penstemons, specially if they're growing in light soil, need watering, so it may be that yours have browning leaves because they went short of water. Its usual to deadhead, but only cut out the flowered stems - later, as winter approaches, its important to leave the topgrowth in place. Many varieties are not fully hardy, and leaving the topgrowth in place helps to protect the roots. Cut back hard in spring, as growth begins.

I was told to cut them back after a first frost

Hmm, interesting, who told you that, booboosmur, and was the advice variety specific? Either way, the advice I've given above is from the RHS...