Do you (or your customers) run backups and have the "Suspend domains until the backup task is completed" option set?
This option causes two restarts of the Apache2 service for every domain to backup up, and if you have a couple domains and each restart takes 5-10 seconds, it can easily accumulate to longer downtime.