On the moment the suggestion to solve the issues with cows not going to the field and not returning is by a small opening in the fence. However the cows will run away and miss Bear enjoys the feast.
So I’ve tried a different solution: I’ve put the barn next to the road, the field on the other side of the road and the fence, put them around the field, barn and road, but put the fence gate on the road where the fence meet the road. This solved all the issues with the barn for me. I hope it will also help others.