With regard to the euphorbia, that is fully hardy in the UK. The delphiniums, though, are another matter - they are hardy, but will be less so because they're in containers and they're susceptible through the roots to cold. You can try insulating the containers with fleece or bubble wrap to get them through the winter. If the boxes are on a windowsill facing south, in the usual situation of house, garden then pavement, road, pavement and house opposite, the microclimate created by such an environment means you're less likely to lose the delphiniums. If the windowsill faces north, it will be much colder. You do not need to worry about a bit of frost - its when temperatures are less than 5 deg C in the day that you have to worry.
And yes, deadhead, water as necessary up to end of October, then check the boxes occasionally to make sure they haven't completely dried out (unlikely unless they're in full sun) but don't water unless essential. You don't want them very wet anyway through winter.

" I am nervous that the forecasts are already talking about early frost, and we are sure to have a cold winter"

What makes you think that? We tend to get the same pattern as NZ and to a lesser extent Australia. Both their winters were one of the mildest yet, so an indication of what may happen for our winter. Canada has had some early snowfall. If the pattern hold then the USA may have another severe winter and us a very mild one.

Didn't you ask this a week or two ago and get the same reply??