should i remove tayberry suckers

Depends on which variety you have. If you remove them haphazardly removing them might cause it to produce thorny suckers as thornless tayberry type is chimeral(outer cells thornless only)and any damage to roots usually produces more suckers. Pulling them out or cutting as close to the main stem is better if these suckers are in abundance.

If you have the medana(thorny one) or any thorny one I would leave those anyway as these are new canes which produce fruit next year, but thin out to 5 or 6 only. In my experience you should never get more than 3 or 4 suckers(canes) anyway so leave them. The old fruited canes you should cut back to about 6 or 9 inches from the ground.