Dig it with a fork when the weather's dry??

I'm trying to create a garden from some derelict land and that definitely has clumpy soil - mainly clay/shale + builders rubbish. I loosen up the top 9-12 inches with a fork and then (on hands and knees) I break down those clumps with a trowel, removing stones and weeds. I then fork in leaf mulch and compost. The section I did 12 months ago was a pleasure to plant last month - no hard clumps.