Leave some until they turn brown and you can 'pod' them and get ripe seeds out...which will also be brown. Then put the seeds into a paper bag and keep them cool and dark until you plant them.

Or remove them altogether as soon as you see them forming and discard, otherwise you will have Perennial Sweet peas all over the place.

That's why I said 'some' Ob! LOL. I cut mine back as soon as the pods form...even before the last flowers have finished.

Well, we ought to, but probably don't