SWAP should be at least half the size of RAM.
If your server makes use of SWAP regularly, you need more RAM. SWAP is an emergency solution for the rare case when RAM absolutely does not suffice. You might see a rare and small SWAP use like 80 KB, maybe 1 MB or so, that'll be alright. But when...