Hey Will,
Sorry to hijack the thread, but I have a save with a similar setup to Karvapollo. I have a double wall built with six barns inside the double wall. I noticed a lot more stuttering after building a double wall enclosing the city. Just checked this bug is reproducible by removing the walls and all of the half second stuttering went away, exactly the same behavior as mentioned by Karvapollo above.
Moreover, this save still has this bug present as well: Game breaking bug: villager is "visually stuck" between a double wall but is still working with zero physical travel time - Build 0.7.4. The laborer is still stuck within the walls but still working fine.
I still have the save for the walls present if you’re interested.