Only sand and gravel? Well, you could try. In the wild it grows in a wide range of habitats from moist woodland to dry, open grassland. As long as it doesn't completely dry out it should grow.

But whether it will flower after the first year is a different matter... I'm curious as to why you actually want to grow in just sand and gravel - could you explain Ruth please.

My experience is that will grow anywhere if it gets a chance with feeding and watering.