Yeah pets start out a bit slow. Another reason why it is better to level with Familiar since it is Ranged. They do get better late game though, once you get items and devotions that increase their speed.
Personally I play softcore as I am not a fan of permanent death mechanics. But the build is hardcore viable as I don’t really die except maybe in higher shards of Shattered Realm where you get insane boss combinations and end up getting swapped into Gravathul’s Red balls by Valdaran or something.