I've got some pretty creeping thyme (try Google for lots of pics.) it's neat, has lavender or pink flowers and smells wonderful if you walk on it. If it spreads too much just pull it out.

Stonecrop(s)?

Another requirement is nothing that creeps or spreads underground as I don't want it getting under the paving.

Everything grows. Best to sink Herbs in 4" pots into the pebbles, then trim later on.

If anyone has clay soil problems I can recommend the
keeping of Gerbils. These are little pets that work for me.
They need to have a supply of plain brown cardboard every day, and chew it into tiny pieces. There is no machine that can do this so well. Every Saturday I take out the litter in the old fish tank they live in. Add it to a large box with old potting compost, some Blood Fish & Bone fertiliser and Horticultural Sand. Plus water. Its marvellous stuff.
I am replanting my potted hardy plants in deep holes in the clay, putting two buckets of Gerbil compost underneath and around each plant, water well, and putting some of the clay soil back on top. My Lavender plants really loved this method to change the content of the soil, and are thriving and flowering. Just 10 mins
Gerbil care daily, cost about £3 a week includes sundries. Result happiness.
Any questions I am quite happy to answer p.m.

I don't know whether you're saying you don't want blue/lavender flowers, or whether you do, but Sisyrinchium bermudanium, if you can lighten the clay soil by adding something to it, and Armeria Pseudarmeria, the Ballerina series, might be a better choice than ordinary Armeria - they're a little bit taller and they have a very long flowering period, longer than Armeria. Otherwise, Ophiopogon nigrescens would look good in pebbles. An afternoon's sun only in summer might not be enough for these types of plants though... they prefer an open aspect with as much sun as possible year round, although the Ophiopogon will do well without that. Carex Evergold, as the soil is heavy, is an option, but all plants do eventually spread, even if they're not sprawlers, it being the nature of growing things to get larger or wider.

Thanks Bamboo - yes I want pale blue/lavender/pink/white flowers - but not shrubby plants.
I was thinking about ophiopogon last night, as it will spill over onto the black paving - I think there's one in particular that is a very slow grower. I don't mind that things will get bigger above ground (within reason) but I really don't want anything that spreads underground and pops up yards away from the original (as bamboo).

That's useful to know about the Armeria Ballerina.
I don't think I'll need more than three plants, just to break up the line where the pebbles meet the paving.