How do you remove one of those "explore here" banners?

I accidentally set one up near a wolf den. One by one my villagers are trying to obey me and getting killed and I have no idea how to stop them now!

select the flag and remove it

For some reason that little dialog box didn’t pop up when I tried it before. Fixed now, thanks!