patch #55 problems

[jprince]

Hello.
At 03:38:33 this morning patch#55 was auto applied.
After which, we observed permission problems.
Local user could not login.
Only root could login.
MySQL could not write to socket.
Postfix died, could not write to file system.


OS: Centos 6.4

--john

 

[jprince]

PSA Log:
127.0.0.1 admin [2013-06-28 03:38:33] 'New patch installed' ('Update version': '53' => '55')

Mail Log:
Jun 28 03:38:32 ihs3 postfix/smtpd[32127]: fatal: host/service localhost/12768 not found: Name or service not known
Jun 28 03:38:33 ihs3 postfix/master[31944]: warning: process /usr/libexec/postfix/smtpd pid 32127 exit status 1
Jun 28 03:38:34 ihs3 postfix/smtpd[32184]: warning: 127.0.0.1: hostname localhost verification failed: Name or service not known
Jun 28 03:38:34 ihs3 postfix/smtpd[32184]: connect from unknown[127.0.0.1]
Jun 28 03:38:34 ihs3 postfix/smtpd[32184]: fatal: host/service localhost/12768 not found: Name or service not known
...
...
Jun 28 03:39:08 ihs3 pop3d: chdir "/var/qmail/mailnames/xxxxx.com/xxxx": Permission denied
Jun 28 03:39:09 ihs3 pop3d: IMAP connect from @ [::ffff:174.225.196.39]chdir "/var/qmail/mailnames/xxxxx.com/xxxx": Permission denied
Jun 28 03:39:09 ihs3 pop3d: Connection, ip=[::ffff:174.225.196.39]
Jun 28 03:39:10 ihs3 pop3d: IMAP connect from @ [::ffff:174.225.196.39]chdir "/var/qmail/mailnames/xxxxx.com/xxxx": Permission denied
Jun 28 03:39:21 ihs3 pop3d: Connection, ip=[::ffff:96.2.33.61]
Jun 28 03:39:21 ihs3 pop3d: IMAP connect from @ [::ffff:96.2.33.61]chdir "/var/qmail/mailnames/xxxxx.com/xxxx": Permission denied
Jun 28 03:39:22 ihs3 pop3d: Connection, ip=[::ffff:24.111.111.222]
Jun 28 03:39:22 ihs3 pop3d: IMAP connect from @ [::ffff:24.111.111.222]chdir "/var/qmail/mailnames/xxxxx.com/xxxx": Permission denied


After restart, MySQL error:
130628 07:27:49 mysqld_safe Number of processes running now: 0
130628 07:27:49 mysqld_safe mysqld restarted
/usr/libexec/mysqld: Table 'plugin' is read only
130628 7:27:49 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
/usr/libexec/mysqld: Can't create/write to file '/tmp/ibCrAZHL' (Errcode: 13)


sw-cp-server log
Note: mod_fastcgi time is GMT, not local..

2013-06-28 08:38:28: (server.c.1543) server stopped by UID = 0 PID = 1871
2013-06-28 03:38:32: (log.c.166) server started
2013-06-28 08:45:22: (mod_fastcgi.c.977) bind failed for: unix:/usr/local/psa/tmp/sw-engine.sock-0 Permission denied
2013-06-28 08:45:22: (mod_fastcgi.c.1171) [ERROR]: spawning fcgi failed.
2013-06-28 08:45:22: (mod_fastcgi.c.3644) all handlers for /login.php3? on .php are down.
2013-06-28 09:40:25: (mod_fastcgi.c.977) bind failed for: unix:/usr/local/psa/tmp/sw-engine-8880.sock-0 Permission denied
2013-06-28 09:40:25: (mod_fastcgi.c.1171) [ERROR]: spawning fcgi failed.
2013-06-28 09:40:25: (mod_fastcgi.c.3644) all handlers for /javascript/promo-flags.js.php? on .php are down.
2013-06-28 09:41:25: (mod_fastcgi.c.977) bind failed for: unix:/usr/local/psa/tmp/sw-engine-8880.sock-0 Permission denied
2013-06-28 09:41:25: (mod_fastcgi.c.1171) [ERROR]: spawning fcgi failed.
2013-06-28 09:42:25: (mod_fastcgi.c.977) bind failed for: unix:/usr/local/psa/tmp/sw-engine-8880.sock-0 Permission denied
2013-06-28 09:42:25: (mod_fastcgi.c.1171) [ERROR]: spawning fcgi failed.

 

[Nikolay.]

Since your installation already seems to be messed up, would you mind rerunning the action that potentially did it and posting the output for investigation?


# ls /usr/local/psa/tmp/MU55-owner_installed.flag
# rm -f /usr/local/psa/tmp/MU55-owner_installed.flag
# sh -x /usr/local/psa/tmp/MU55-change_owner.sh

 

[jprince]

Hi.
We are waiting a full recovery test instance..
Will try it then.

I modified the script to echo instead of execute..
We found a customer generated dir in statistics/logs/2/www.theirdomain.com/products/4 Door Mini Truck
That appears to have caused the problem.
snippet from change_owner script..

  chown root:psaserv /var/www/vhosts/-----.------.com/statistics/logs/2/www.----------.com/products/4
        chmod 0550 /var/www/vhosts/-----.------.com/statistics/logs/2/www.----------.com/products/4
        chown root:psaserv /var/www/vhosts/-----.------.com/statistics/logs/2/www.----------.com/products

        chmod 0550 /var/www/vhosts/-----.------.com/statistics/logs/2/www.----------.com/products
        chown root:psaserv Door
        chmod 0550 Door
        chown root:psaserv .
        chmod 0550 .
        chown root:psaserv Mini
        chmod 0550 Mini
        chown root:psaserv .
        chmod 0550 .
        chown root:psaserv Truck
        chmod 0550 Truck
        chown root:psaserv .
        chmod 0550 .

Script is not handling names with spaces in them..
We frown on names with spaces, however it is tough to enforce..

 

[Nikolay.]

That's what I suspected. But it shouldn't have wrecked your system that much...

Would you mind posting output of following command as well?

# ls -ld / /var /var/qmail /var/qmail/mailnames

 

[jprince]

I do mind,
Rather not disclose that information.

 

[jprince]

Also, those perms look fine..

 

[Nikolay.]

I do mind,
Rather not disclose that information.

This information does not contain anything sensitive. Here's mine

# ls -ld / /var /var/qmail /var/qmail/mailnames
drwxr-xr-x 21 root root 4096 2012-04-26 16:32 /
drwxr-xr-x 17 root root 4096 2013-06-28 14:46 /var
drwxr-xr-x 4 root root 4096 2013-06-28 14:44 /var/qmail
drwxr-xr-x 3 root root 4096 2013-06-28 15:12 /var/qmail/mailnames



If you're paranoid about such simple things I suggest addressing Parallels support instead.

 

[jprince]

Hi Nikolay.
Appreciate the response.
Mine was more informative. Was really not asking for support.

Further, I responded in passing...
drwxr-xr-x. 27 root root 4096 Jun 28 09:55 /
drwxr-xr-x. 25 root root 4096 Jun 10 17:53 /var
drwxr-xr-x 4 root root 4096 Jun 2 21:51 /var/qmail
drwxr-xr-x 254 root root 12288 Jun 18 16:23 /var/qmail/mailnames

thanks.
--jp

 

[Nikolay.]

Mine was more informative. Was really not asking for support.

Makes sense

I'm just curious which permissions got so screwed that you got the errors posted above. Or maybe this script from MU wasn't the culprit at all.

 

[jprince]

Hello.
/ was changed to root/psaserv :0550

Was without a question, the patch..
--john