Perhaps contrast the planting with the formality? I've used geranium 'Brookside', long-flowering, to great effect within a box square. Think, too, on hydrangea paniculata cultivars for a frothy mid/late summer show. The geranium could underplant the hydrangea?
W.

It might be worth your looking on line at Monty Don's garden - he has an area called the Jewel Garden which is low box hedges surrounding brightly coloured planting within, in a boxed format as you're describing. He seems to grow taller perennials in oranges, reds and yellows mainly, to give a bright burst of colour. Check out Rudbeckia, Echinacea, Heliopsis varieties, taller dahlias and so on.

Most herbaceous perennials disappear altogether in winter, but you can interplant them with spring bulbs, crocus, daffodils, tulips etc. Different varieties flower at different times and with care you can cover late December right through to May. Worthy's geranium idea is good - you might also look at Geranium Rozanne, which spreads wide but stays low and flowers for a long time. This would cover the ground in between your taller perennials that Bamboo suggested. But as for low maintenance, you would still have do do the weeding!
Or depending on the length of the plots you could have one two or three specimen shrubs, perhaps formal not too vigorous conifers, and keep the rest of the planting lower?

At Hidcote I saw box hedges with hardy Fuchsias planted inside. It was very effective especially where the arching branches went over the hedges. You could under plant with spring bulbs.