Raiders camp

I thought that following retreating raiders would lead them to their camp but alas they just go to a spot and poof out of existence.

Can we have them to retreat to their actual camp so I don’t have to sweep the cavalry for the entire map pointlessly to find it?

They don’t all necessarily come from camps on your map. While destroying a camp should delay you getting any attacks for a while destroying all the camps on your map won’t stop them from continuing to raid you.