This year we think we found the culprit.
A Merlin. This zoomed across the field at the other side of our fence, so we couldnt even count that as it wasnt over our garden. As soon as it disappeared - and our hour was up - all the garden birds returned. Sheesh.
While we are on the bird subject. Bit if a cheat on the sky only picture, but ho hum, a murmuration of starlings.
Hope your garden was full of birds today.