I can see that you have planted the Betulas as a screen and there are three things to take into consideration. 1 is the shade aspect. 2 is the fact that whatever you plant, they will be competing with the trees for moisture and 3, you are playing games in the garden so anything of any size will be damaged by a ball. So, although not the ideal, Vinca Minor will spread as ground cover over the bare soil and fill the site.

Thank you. I will take that into consideration. Pittsporium not an option then. It looked so pretty.

One of the features of your Birch is the whiteness of the trunks - if you plant something that gets up to 4-6 feet, it would block your view of them and will, as Jimmy says, compete with the trees for resources. If you like neat growing plants that make neat shapes, I would not recommend Vinca as ground cover - it sends out long growths, leaving bare gaps inbetween and sprawls everywhere. You'd probably prefer the growth habit of a particular variety of Campanula - C. portenschlagiana, previously called Campanula muralis - it doesn't mind a fair bit of shade but also does well in full sun. It makes neat, evergreen, rounded, low hummocks that don't sprawl (unlike C. carpatica and other varieties).
If the area gets sunlight for half a day, you could use Hebe 'Green Globe', which makes a rounded, neat shape, eventually up to 15 inches wide and high with no need for pruning or clipping, but stick to 2 or 3, maybe 4 max, arranged along the front, so as not to compete too much with the trees. Maybe fill in between with the Campanula muralis for groundcover. If it's pretty shady a lot of the time, a particular lilyturf, Liriope 'monroe white' tolerates a lot of shade and still flowers - makes grass like clumps which are evergreen, flowers white in autumn.

If you want the effect of Pittosporum there is a very dwarf neat Berberis that might fill the bill. Whatever, you will need to widen the border or mowing to the edge of the lawn will be a nightmare. Have a look at Berberis atropurpurea nana compacta - you can get them with either red or gold leaves. They aren't evergreen but you can always but some spring bulbs in between them.
Same spacing as bamboo's suggestion for hebes.
As its a small garden you could get some height by stringing wires on the fence and trying a clematis (some will flower twice in the year) but you'd need to shade the roots. Or using the same idea try climbing Rose Casino which doesn't mind some shade.

Hope one or two of all these ideas will appeal to you!

Take a look at Ajuga reptans 'Black Scallop. The shiny dark leaves will offset the silver birch. It's a low growing, weed smother habit that is very eye catching and gives a finished apperence. It sends up loads of blue flower spikes in the Spring that the bees & butterflies love.

PS My last suggestion was for the fence along the bottom, not under the Betulas. Planting under them you need to remember that they are very shallow rooted so the ground will be dry.

My choice would be box balls, box does well in shady situations, for good effect I would not over do it, and would plant no more than say four of these I would zig zag through the birches with one in the centre of trees, this will give you a good effect, and the contrast in the colours throughout the winter is fantastic, what you don’t want to do is let the box balls get to oversized for this small area, birch have lots of surface roots so if you do choose box then be prepared to prune a few roots away to get them in, be aware of box eating catapiller and box blight, you could opt for using lonicera baggesons gold instead, very quick to get going but will need a few trims in a season to keep them compact.