Sometimes excavating the soil as far as possible to a good spade's depth or more, then levering it out with something like a scaffold pole works, specially if the ground is nice and moist. Otherwise, its a stump grinder which can be hired (hard work) or you pay someone to come in with a stump grinder to remove it and take the debris away.

I've tried digging round it as deep as I can and then sawing off all the roots I could find until it could be lifted out. This leaves lots of old roots in, which sometimes have a go at growing again, especially sycamore will do this, and a horse chestnut had a go but was easily discouraged.

It also depends on what your definition of small is. For me, that means anything less than 5 cm in caliper, since that can usually be pulled up with human power, assisted by simple tools. From 5 cm to 25 cm, it can still be pulled up--partly depending on soil conditions--but power equipment is necessary. Above 25 cm, digging and cutting roots, by hand or with power equipment (including grinding), is needed, with all of the attendant risks of suckering and soil cavities.