Yes, some sets are restricting that way, similar to SR set, but not all of them. Justicar is prime example of the set used by many builds in different number of items. Vileblade is easily used without the weapons. Lots of sets have neat +120% to dmg for 2-piece bonus.
Shattered Guardian, as a guaranteed drop, is the “main” set of the game and the fact that it promotes uncreative “all or nothing” is bad.