When going from a resource node/pile to a work site, Villagers have this habit of taking the shortest possible path to their destination. Which is great, usually, except when that path takes them right through the middle of a raider camp.
This screenshot shows the camp fully discovered a small distance from my town. Even though it is discovered and my villagers are ostensibly “aware” of its presence, they still take the shortest path through it. Sure, maybe I should not be building a wall on the opposite side while there is a raider camp nearby but I would hope that my villagers have the presence of mind to avoid the shooty, stabby guys sitting two doors down. I would at any rate.
Edit: Removed rant about camp difficulty. After getting good, I was able to take it out by fighting on terms more favorable to my success rather than charging headfirst into the enemy camp. Not exactly the most Democratic move, but you do what you gotta do.