What I normally do is I complete the main campaign for veteran and elite
without doing AoM. Then when I reach ultimate difficulty I do AoM since I
normally like hanging around ultimate more than any other difficulty. I’m
actually in the same boat as you with my gunslinger build. I am level 56 and
just reached Fort Ikon, “Yes, I use Potion of Clarity a lot.” But since I am
already over leveled for most areas due to the Potion of Clarity, I may just
jump straight into elite after completing everything in the base game. There
comes a point where you will be too high of a level for areas in veteran if you
hang around too much trying to farm and gear up for elite. This is the
problem with AoM in the beginning difficulties since it does just that. I would
recommend starting elite because you will have a better chance at finding
purples and better loot for resistance stacking than compared to
veteran/normal. I would also prefer to do AoM in elite since ultimate is where
things actually get difficult. Veteran is naturally harder than elite because of
the not so good loot drops when in comparison to the elite loot drops, FYI.
With that being said, you are the perfect level to start elite.
My Personal Strategy for the Playthrough Difficulties:
Normal - Noob 
Veteran - Full map clear, shrines, quests and dungeons. No AoM.
Elite - Not a full map clear. Bum rush to the shrines and quests. Do a dungeon or two if really needed. Complete AoM or not and do a resistance check.
Ultimate - This is where you will spend most of your time with your character gearing up and farming your brains out. Prepare for fun lol.