Or is it pick the highest debuff? if it stacks i have a few ideas… 
i believe it works like RR, -DA stacks, but reduced DA only the highest reduction is applied.
Stuff like -8 Defensive Ability stacks with others like it. Stuff like Reduced Target’s Defensive ability by whatever points doesn’t stack with others like it and pick the highest.
These two types do stack with each other.
And to add onto what Norzan said, any “-x <stat>” stacks. The way I like to think about it is a debuff forces mobs to wear a piece of armor except its stats are negative (resistance, OA and DA, attack speed…).
so in other words Bile Fiend’s ability Rotting Fumes cannot stack with Inquisitor’s Biting Cold modifier for Rune of H. Because those state “### Reduced Target’s Defensive Ability”
Pretty much, yeah.
A rule of thumb in GD - buffs/debuffs from SAME source dont stack, the most powerful effect takes place. Buffs/debuffs from different sources stack with each other.
Debuffs from several applications of the same skills dont stack with each other, because they’re considered “same source” (for example 2 Curse of Frailty from 2 players wont stack with each other).
Debuffs with “X reduced <stat> for Y seconds” place an indepent debuff for listed duration. All such debuffs, that reduce the same stats, are considered the SAME source in GD, so they dont stack with each other.
Debuffs with "X% reduced <stat> for Y seconds place different debuff compared to “X reduced <stat> for Y seconds” for the same <stat> so those 2 stack with each other.
Effects with “-X <stat>” are tied to their mother debuffs, so if debuffs come for different source (different skills/items/etc), they stack with each other, even if <stat> is the same.