You have done so well to get a litchi that far. I've tried repeatedly to grow Litchi from seeds (from the fruit available at Xmas time) but as soon as the little trees reach about six inches, they've just died, despite being into warm weather and in a greenhouse. I've never been able to find out why.
Did you buy yours as a grown seedling or produce it from seed?
In general terms, why trees drop fruit can be explained by the following:
(Our Feijoa or Acca did the same last summer)
1. Tree is too immature to maintain fruit to maturity. It will fruit successfully when older and larger.
2. Irregular watering. Once a tree is stressed, the first thing it does is drop its fruit.
3. The flowers are not properly pollinated, so the fruit is not viable. Some fruit trees need cross pollination even though they appear to self-pollinate.
I would suspect that irregular watering if your tree is in a pot, or dry soil if it is planted in the ground.
I'd suggest you do some research on litchi cultivation on the Internet which may give you some more help.
Beautiful tasting fruit, good luck for this year.

This website has lots of useful information about growing litchees.

http://www.safruitfarms.com/Default.aspx?tabid=410

Many thanks for your answer. The tree was purchased as a young sapling and planted in a huge hole filled with acid compost. It has regular irrigation periods but I was wondering if it needed extra water during the fruit set period. I will try the website you gave me, once again many thanks for your help.
Badboyroy