If you build a new farm, you can graze cows on it for fertility gains. But if you use the “Expand Crop Field” it creates a permanent spot in that old farm grid where cows can not graze. Even if you remove the farm completely, that area is permanently blocked out. Only happens when you use the Expand Crop Field.
Edit: The blocked area disappeared a few months after removing the farm, just took some time to update I guess. But if you keep the expanded farm, that area is blocked off from grazing forever.
Edit 2: So I reloaded the map and now the two farm areas that were never expanded are red and blocked from grazing. Something is really wrong here, there was no blocked areas, I reload and now there is.