The answer is absolutely yes, it is possible to even beat the odds x2 or more.
Here is a video of 2588 invaders attacking my settlement with a population of 2024.
Note the only mod that is allowed is: Tower Defense to help increase the number of invaders
