Play around with the calculator to see to what degree the returns are diminishing. Average damage is a marker for your DPS (what % of your damage you deal on average with a given OA against a monster with some DA).
Probably only a couple very specific builds (I’ve made one like that recently). Also might depend on your endgame activity as superbosses have more DA. Usually you have more important things to worry about. Also you need high % Crit damage to start considering it.