Domain aliasing or redirection is certainly possible using mod_rewrite, that is one of it's functions. But again, with 50 active and the other 3950 aliases, you still have a lot going on for a somewhat limited VPS.
Just because the others are redirected or aliased, does not mean they don't count. You could have a single active site and all other 3999 redirected/aliased, they still will add to the overall load or hits in any case.
Not only that, but just writing the 3950 rewrite statements will be a lengthy project in any case.
My last words on this matter are: If you are unwilling to take the advice that your server is a bit underpowered for the task, then I guess you will just have to keep suffering.
And Whistler is correct about Apache tweaking or performance tuning. This is an additional procedure, but really won't do much on your current server IMO.