Instead of relying on the unreliable “traveling salesman” trading system, make Trading Outposts into nodes for a map-wide trading network where the player can decide what and where to transport goods. This would allow for much wider opportunities of exploiting the resources on the map. You could set up a fishing village and send the food across the map to a mining outpost, and send the ore to your city center etc without having to establish major infrastructure everywhere.
With greater access to internal map resources the need for an external random source would also be greatly diminished, and organizing and maintaining your internal trade would be a far more engaging system than the current one. It could also open new dynamics like raider ambushes with solutions like designating road patrols or caravan guards.
Personally I find the idea of managing a regional area filled with villages, outposts and cities far more appealing than the current focus on a single settlement. It would even be helpful for better optimization within major cities.