If you look at Barrels in the “Settlement Items” window, it lists the places that it is “Used By”. The list in incomplete. It does not list the Market or the Granary. I could not work out where they were going for a moment.
Also posted on the Steam message board but was directed here.