You have 4 workers whose job is to stock your houses with food, necessities, and luxuries. If they can’t keep up with that job (forcing villagers to stock their own houses), then 99% of the time the problem is that there are too many houses for them to stock.