Hard to say for sure, don't know how you've been caring for them, but certainly the one with more green leaves looks as if its had or got powdery mildew, which often indicates dryness at the root. The two newer leaves look clear of it though. Lupins also want to put down a long tap root, and they can't do that in small pots, that might be a contributory factor.

Also, what kind of potting compost is under all of that gravel?

Compost is a mix of home made and bought from garden centre. Shall I pot them on? Is it too late to sow some more seeds. I have greenhouse.

What did the bag from the garden center say? Some commercial composts are intended for soil conditioning only, and not for potting.