I’m pretty sure this is a bug because the ore is in fact accumulating in the stockyard right next door.
Same with my clay pit which gets frequent visits from a wainright. The miners always say they’re gathering ore when I select the building.
I’m currently loaded with clay from a previous site, and that one also displayed similar stats.