Early potatoes take around 12 weeks from planting to being ready to eat. Main crop take even longer, I would not normally expect to be lifting early potatoes until end of June beginning of July. It may well be later than that this year due to the long winter.

You can dig around the stem of the plant with your fingers to see if you can see/feel any tubers forming and how big they are.

could be they are main crop potatos and you have dug them up to soon