SSL on email broken since 11.0.9 MU#62

[shall]

# cat /root/.autoinstaller/microupdates.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<patches>
    <product id="plesk" version="11.0.9" installed-at="20121209T212320">
        <patch version="62" timestamp="" installed-at="20140723T035123" />
    </product>
</patches>

Minutes after this microupdate auto-installed, I am unable to login to POP and IMAP on this server. None of the accounts are able to login. The logs indicate hundreds of login failures, accompanying a new record I've never seen in these logs:

pop3d-ssl: Unexpected SSL connection shutdown.

Unfortunately, I can't find anything about what MU#62 actually included. The changelog only indicates up to MU#61, so I'm very concerned:
http://download1.parallels.com/Ples...-11.0-linux-updates-release-notes.html#110962

I've googled the specific error I'm seeing above, but nothing appears to be remotely related to this problem.

Does anyone have any recommendations?

Is it possible to roll back the MU to determine if that is in fact the problem?

 

[shall]

Here's the details on MU#62:
http://kb.parallels.com/en/122245

Unfortunately, it doesn't look like this should be causing this. Any other suggestions? I'm going to run mchk now.

 

[shall]

mchk reported that mail_auth_dump failed. I went through the processes here to fix it:
http://kb.parallels.com/en/121095
There were a couple forwarder accounts without passwords assigned. These were fixed. No other problems identified.

Any thoughts or input on where to go from here?

 

[shall]

Parallels logged into the server and "fixed it" by reverting to the pre-upgrade database, populating the email accounts with no passwords (such as forwarders) with a generic password, and then converting the mail server to use qmail instead of postfix.

This resulted in many scripts and SSL-capable SMTP failing completely. They then reported that it was "fixed" and that it was not caused by MU#62.

First, it is not "fixed" if the ability to send mail is gone. Second, MU#62 did, indeed, cause the problem as the logs became flooded with login failures literally seconds after MU#62 was installed. They claim the login errors only came from two IP addresses, but that's a bald-faced lie. I have copies of the logs which indicate otherwise. The day before MU#62 there were 7587 successful email logins (from 135 different IP addresses). The day of the MU#62 there were only 38 email logins -- and all from the two hours preceding the MU#62 installation. Since I *actually* fixed it a few hours ago (several hours after Parallels claimed it was resolved) there have been 754 successful email logins. When mail was broken, the day there were only 38 logins, the number of distinct IP addresses that suffered from the "Unexpected SSL connection shutdown" error was 26 (not two, as they claimed).

The actual cause appears to be a coding tolerance problem that causes issues whenever third-party yum repositories are enabled. This is detailed in this KB article:
http://kb.parallels.com/en/117910

What it doesn't mention in the article is that leaving third-party repositories enabled when Plesk auto-updates are also enabled (such as micro-updates) can result in a condition that leaves the server in an unusable state. Oops. Kind of a big thing to leave out.

It's almost as bad as the previous revision of KB #121095 which told me to run a script to reset passwords that they claim is the "real" reason why emails were broken. They didn't just not fix it, they had the audacity to blame me for following their instructions to fix it --- which broke it even worse. Then lied about what the log files actually said. Sigh. Disappointing to say the least.


At least it's working now. For anyone else that experiences random problems after a micro-update, FIRST make sure that you don't have any third-party repos enabled.

ls /etc/yum.repos.d

You should be able to just rename their extension from ".repo" to ".repo-old". This should make it easy to re-enable them again when you need them.

Next, restore your pre-update databases (you'll find them at /var/lib/psa/dumps/mysql.preupgrade.*). You'll want to extract only the 'psa' portion of the dump file (that is, everything from the top down to where you see the second instance of "-- Current Database:"). Save that portion of the file as something like "dump.sql" and then run:

mysql -uadmin -p`cat /etc/psa/.psa.shadow ` psa < dump.sql

Finally, run the autoinstaller again to finish the patch that failed before. You can run the micro-updater with this:

/usr/local/psa/admin/sbin/autoinstaller --select-product-id plesk --select-release-current --reinstall-patch --install-component base

But I recommend you run the complete installer to ensure that various components are how you want them configured:

/usr/local/psa/admin/sbin/autoinstaller