Plesk 12.0.18 Update #33 breaks MySQL on CentOS 5

[burnley]

Hi folks,

Running the latest Plesk MU patch today I discovered that Parallels has replaced our
*working* and *properly packaged* MySQL packages with their own.
Before:
mysql-5.5.41-1.el5.remi
mysql-bench-5.5.41-1.el5.remi
mysql-libs-5.5.41-1.el5.remi
mysql-server-5.5.41-1.el5.remi
After:
mysql-5.5.41-15012715
mysql-bench-5.5.41-15012715
mysql-libs-5.5.41-15012715
mysql-server-5.5.41-15012715

There are several issues here:

1. Critical! Never, ever, overwrite /etc/my.cnf on a configured system! There are customizations in there which, if you remove, will render MySQL unusable! For example, I had this variable set in /etc/my.cnf:
innodb_data_file_path = ibdata1:170M;ibdata0:2048M:autoextend
Guess what happens if you remove it and restart MySQL? Does this look familiar?
150130 10:39:09 InnoDB: highest supported file format is Barracuda.
InnoDB: Error: trying to access page number 14916 in space 0,
InnoDB: space name ./ibdata1,
InnoDB: which is outside the tablespace bounds.
InnoDB: Byte offset 0, len 16384, i/o type 10.
InnoDB: If you get this error at mysqld startup, please check that
InnoDB: your my.cnf matches the ibdata files that you have in the
InnoDB: MySQL server.
150130 10:39:09 InnoDB: Assertion failure in thread 140490375583520 in file fil0fil.c line 4578
InnoDB: We intentionally generate a memory trap.
[...]

2. Major! Do not forcibly obsolete existing packages unless you have a *VERY* good reason to do it! On our CentOS 5 and 6 servers we are using 5.5.x MySQL packages from the remi repository, which are known to work and update properly, were thoroughly tested for a long time and are trusted by quite a few sysadmins around.

We can not continue applying this patch on all our remaining Linux Plesk 12 servers. You must fix this issue by making sure you are maintaining backwards compatibility on all the systems running already the latest 5.5.41 rpms. Overwriting /etc/my.cnf was a very poor decision, if you want to install your own version please follow rpm packaging best practices and install it as .rpmnew!

Parallels, you've just ruined our Friday here!

 

[burnley]

Quick update, in case someone else will be hit by this problem. I found a way to prepare the system for the mysql takeov^H^H^update:

mkdir -p /etc/my.cnf.d
cp /etc/my.cnf /etc/my.cnf.d/

On the assumption that the new /etc/my.cnf will contain the line "!includedir /etc/my.cnf.d" the disaster should be averted.

 

[abdi]

Hello Burnley,
That's really unfortunate!
A word of advice in future before running any server upgrades, its always recommended to run a system backup of your server especially the /etc folder. You know anything can go wrong ..(some times even in the most UN-expected situations). A single file failing to downloaded could cause a huge disaster to your server..

 

[burnley]

Hi abdi,

We keep up to date backups of all the critical files on our systems, this is sometimes what gets us out of trouble But this is not the point here: I'm strictly arguing about the need of doing the package management properly. As a software provider you can't just stick your boots into the clients systems because you can, you've got to do the right thing and follow the best practices. Don't get me wrong here, since I started working with Plesk, more than 4 years ago, the Panel has come a long way and it's much more mature now, but Parallels have got to be careful with the software change management because the bar is set pretty high now.

 

[UFHH01]

Hi burnley,

please consider switching OFF the automatic update system for Plesk and probably choose the ( not from Parallels recommended ) manual option. As a longterm user of Parallels software, I don't think that unattended updates/patches/upgrades are a good idea for production servers, especially not, if you modified configuration files for several services. This will result in a little bit more administration work for your servers, but this one does not renounce the possibility to read the change logs, before any patch/update/upgrade and will reduce possible necessary investigations and as well "ruined fridays". "Critical updates" will still be installed automatically, but from my longterm experience, these critical updates ( mostly ) never impair the running systems.

 

[abdi]

Burnley,

I know this is not the best advice (till the developers sort that out, if they ever) but I normally use this procedure in such situations:
I lock the files I don't want any one or any program to update. I use

chattr +i filename

.
And when am ready or want to make changes to the respective file, I then unlock it with

chattr -i filename

Hope that helps!

 

[burnley]

UFHH01, we don't do automatic updates. This particular MU patch contains a different set of mysql 5.5.41 rpm packages to the ones we were already using. Why did Parallels decide it's such a wonderful idea to push these packets on a perfectly fine, stable system that was already running 5.5.41 from a very reputable source? Don't know.
abdi, did you test it with Plesk? I don't forcing the immutable flag on files modified by Parallels is at all safe, can you guarantee a predictable behaviour in case the autoinstaller will not be able to write in an immutable file?

 

[IgorG]

Guys, we have checked this issue and submitted corresponding bugreport PPPM-2480.
I hope it will be fixed soon. Thank you for cooperation.

 

[burnley]

Thanks Igor. Hopefully the fix will be backwards compatible with the existing running environments and won't cause new issues on CentOS 6 and 7.