You don't say which part of the UK you live in, but here goes anyway! The first one that springs to mind would be a fruiting Holly. This could also have variegated foliage. More research would be needed to identify the right variety to choose, bearing in mind also the growth rate and shape required. Or a Eucryphia or Magnolia grandiflora, if your soil is acidic, these have white flowers in late summer. No fruit though.

Holly is the only evergreen with autumn fruit that I can think of! If you go instead for a tree with a beautiful shape when bare you would be surprised how much pleasure it can give you especially if you go for one that has good autumn colour or spring blossom. There is always something to look forward to with these trees, whereas holly is always more or less the same apart from the berries, which the birds soon eat anyway! There is a tree sized cotoneaster - don't know the variety- that has gorgeous autumn berries and holds its leaves quite well, but you couldn't call it really evergreen.

There are many beautiful small trees that do fruit well, and a fair few that are evergreen, but temperate-hardy evergreen trees that produce autumn fruit are few and far between, I'm afraid.

It's worth remembering that no female Ilex (Holly) will bear fruit unless there is a male in close proximity.

If you are in a warm part of the country then Cornus capitata might be a good option, and your sheltered, south-facing site would suit it too.

There are several truly evergreen, and tree-sized Cotoneaster species now available which do fruit prolifically in Autumn including C. glabratus, C. glaucophyllus, C. henryanus, C. marginatus, and C. salicifolius - this last being quite widely available.

Arbutus are evergreen with autumn fruit as well as attractive flowers and beautiful bark, but it will take a while before they reach 6 metres.

You would widen your options considerably if you could plump for either evergreen or fruit-bearing. Also the best and most garden-worthy trees have interest all year round, be it from bark/flowers/fragrance/foliage/fruit, so it's good to consider what the plants would look like in Spring & Summer, as well as Autumn, for instance.

Below is a good article on small-ish trees for the garden that may give you some further ideas:

https://www.ecocharlie.co.uk/blog/2010/02/trees-for-the-garden/

Another possibility is amelanchier 'Ballerina'. It carries white flowers in the spring (mine is just coming into flower now) with foliage that starts bronze but quickly turns green. It carries berries in August that the birds love (so they don't last long unfortunately) and then you get autumn leaf colour later as well