My fig is in a pot and all the young figs fell off early this year, which I assumed was because of the freezing cold temperatures. I've read that they need plenty of watering in order to retain the figs in spring. Someone here will know more about them than I do.
I also remember reading that if the little figs are only pea-sized at the end of summer, we may as well take them off because they won't develop enough to become adult figs.
I had a good crop last year and the previous winter had been pretty bad at times so my thinking about freezing temps could be totally wrong. Gardening is pretty complicated! Be interested to see what other members think.

You should remove all the unripe figs at the end Summer that's now. The little tiny ones as well, even if they make it through the winter, as they will not fully ripen. Taking all the fruit off stimulates the tree to produce new fruit early in the Spring which then get a full season to ripen.

If you leave fruit on, the tree wastes energy on fruit that stay unripe with thick skins or drop off later in Spring and Summer. It then produces new fruit but there is not time for it to ripen.

I hope that makes sense

Thanks foryour usful help