How to de-select a villager?

I am going to ask here first, assuming that I am dense and there is a way to deselect a villager or villagers after an area-drag selection of multiple villagers?

I know I can reselect a subset more carefully, but the initial selection will still have the task override set and I have to move them to a nearby area, to their current location, and then wait out their “Awaiting Command” timer before they go back to their usual task. This can take the better part of a season and can even do things like make crop harvest not happen.

Or is there a way of having a villager return immediately to their task without the “Awaiting Command” timer?

My first random maps have been mountains and sometimes it selects the villagers on the other side of the hill that I can’t see … until I see them come running …

Left clicking on the ground deselects whatever you had selected, including villagers.

I have not found a command of “nvm go back to normal”, but the timer is pretty quick, so once I’m done with a villager I just tell them to… walk a foot from where they are so they are not stuck in some long-distance trek, and leave them alone.