Maybe like the sweets called "lemon drops" your tree is a distant relation.

Mine do that, we find they are better outside in a shady area and kept moist but not wet. The leaf drop is worse in side in the winter even though we mist them

Citrus plants need a humid environment and so keeping them indoors isn't a good idea. If the leaves are a pale green or yellow this could indicate a lack of nutrients and would also cause leaf drop.

Poor drainage or over-watering would cause the roots to rot so that the plant doesn't get enough water and food.

Another cause could be spider mite but you should be able to see them.

So,

Try and keep the plant outside in full sun.

If you need to over winter then the best place would be in a garage.

Feed with a citrus plant feed This is especially formulated for citrus and contains the right balance of fertilizers and trace elements.

Use a well drained soil and don't over water.