• If you are still using CentOS 7.9, it's time to convert to Alma 8 with the free centos2alma tool by Plesk or Plesk Migrator. Please let us know your experiences or concerns in this thread:
    CentOS2Alma discussion

virtual.db workaround code

J

jandrews

Guest
Guys,

After upgrading to 9.0.0 on Debian Etch x64 I encountered numerous probs, here are the ones I havent seen a fix for yet:

1) Aliases using reserved system aliases such as "info", would not deliver.
2) Some mailboxes were not being recognised, deleting the mailbox and recreating fixed it, but I have too many to do this manually
3) Email originating locally (eg from PHP forms) were not relaying in instances where I was not handling the MX.

Rather than wait for Parallels to fix it up, I have written my own work-around that cleanses and fixes the virtual.db.
If found that Plesk adds the following aliases automatically WITHOUT checking if there are conflicting aliases created by the customer:

abuse
postmaster
root
support
anonymous
(also a number of mailman required aliases).

I also noticed these aliases get created regarless if one of the MX records match one of ours. If we are not handling MX at all, they still get created :\

User defined mailboxes dont get a local delivery rule if the name matches a reserved alias.

My script fixes all of these issues by running every hour, it also removed the mailman reserved aliases in instances where a domain doesnt have a mailman list (active or inactive). Only if a problem is found will it finish a rebuild. If a prob is found you get emailed a report with what has changed.
I provide the script with no warranty whatsoever and urge only those who know what they are doing to attempt to use my code, everyone else wait until swsoft release a fix (im not holding my breath).

Here we go........


/usr/local/plesk9_workarounds/fixPostfix.sh (dont forget to set the execute perm):
#!/bin/bash
/usr/bin/php -d safe_mode=Off -d open_basedir="/" /usr/local/plesk9_workarounds/fixPostfix.php




/usr/local/plesk9_workarounds/fixPostfix.php:
#!/usr/bin/php5 -q
<?
/* 15/01/09 Written by Jeremy Andrews
This Software is provided "AS IS" and "WITH ALL FAULTS," without warranty of any kind, including without limitation
the warranties of merchantability, fitness for a particular purpose and non-infringement. I make no
warranty that the Software is free of defects or is suitable for any particular purpose. In no event shall I be
responsible for loss or damages arising from the installation or use of the Software, including but not limited
to any indirect, punitive, special, incidental or consequential damages of any character including, without limitation,
damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses. The entire risk as to the quality and performance of the Software is borne by you.
Should the Software prove defective, you and not I assume the entire cost of any service and repair.
*/


# Config #####################################

$TEST_MODE = TRUE; # Wont write any config files, just echo output to stdout.

$db_host = "localhost";
$db_login = "xxxxxx";
$db_password = "xxxxx";
$db_name = "psa";

$postmap = "/usr/sbin/postmap";

$our_mx_servers = array("",""); # Flat array of our mx servers.

$file_path = "/var/spool/postfix/plesk";
$tmp_dir = "/tmp";

$mail_report = TRUE;
$report_email_address = "";

##############################################

# lets spin through the output of our postfix db file.
exec($postmap." -s ".$file_path."/virtual",$output);

$db = mysql_pconnect($db_host,$db_login,$db_password);
mysql_select_db($db_name,$db);

$body = "";
$changed = FALSE;
$removed_domains = array();

$output_buffer = "";
while (list($key,$line_raw) = each($output)){

$line = explode("\t",$line_raw);

# Determine the domain in question.
if (substr_count($line[0],"@") == 0){
$domain = $line[0];
} else {
$d_array = explode("@",$line[0]);
$domain = $d_array[1];
}

if (in_array($domain,$removed_domains)) continue;

# If we are NOT handling the MX for this domain, we need to ommit it from our table.

$local_mx = FALSE;

$result = mysql_query("select val as mx_record from dns_recs where type='MX' and displayHost='".$domain.".'");

while ($row = mysql_fetch_assoc($result)){

# Is it one of ours?

reset($our_mx_servers);

foreach($our_mx_servers as $our_mx_server){
if ($row["mx_record"] == $our_mx_server.".") $local_mx = TRUE;
}

}

# None of the set MX records match our ones.
if ($local_mx == FALSE){
$removed_domains[] = $domain;
$changed = TRUE;
$body .= "We are not a listed MX for [$domain], removed.\n";
continue;
}


if (substr($line[1],-22) == "@localhost.localdomain"){
# Arrh bugger, where is it supposed to go?

# Is it a mail account?
$result = mysql_query("select redir_addr from mail left join domains on mail.dom_id = domains.id
where concat(mail_name,'@',name) ='".$line[0]."' limit 1");

# Modify our line
if (mysql_num_rows($result) > 0){

$body .= "Found mailaccount line [".$line[0]." ".$line[1]."] changing to [".$line[0]." ".$line[0]."]\n";
$changed = TRUE;
$line[1] = $line[0];


} else {

# Is it an alias?

$result = mysql_query("select concat(m.mail_name,'@',d.name) as destination from mail_aliases as a
left join mail as m on a.mn_id = m.id
left join domains as d on m.dom_id = d.id
where concat(a.alias,'@',d.name) = '".$line[0]."' limit 1");

if (mysql_num_rows($result) > 0){

$row = mysql_fetch_assoc($result);

$body .= "Found alias line [".$line[0]." ".$line[1]."] changing to [".$line[0]." ".$row["destination"]."]\n";
$changed = TRUE;
$line[1] = $row["destination"];

} else {

# Does this domain have a mailman list?
$result = mysql_query("select 1 from MailLists left join domains on MailLists.dom_id = domains.id where domains.name='$domain'");
if (mysql_num_rows($result) == 0){
# This domain doesnt have a mailman list, might as well ommit this line.
$body .= "No mailman list found, deleting [".$line[0]." ".$line[1]."]\n";
$changed = TRUE;


continue;
}
}
}
}

$output_buffer .= $line[0]."\t".$line[1]."\n";

}

mysql_close($db);


if ($TEST_MODE == TRUE){

$body .= "\n\n".$output_buffer;
echo $body;

if ($mail_report == TRUE) mail($report_email_address,"##TEST## Plesk Postfix workaround - rebuilding $file_path/virtual.db",$body);


} else {

# Only do something IF an actual change has been had.
if ($changed == TRUE){

$tmp_file = $tmp_dir."/virtual";
exec("rm ".$tmp_file);
$fp = fopen($tmp_file,"w");
fwrite($fp,$output_buffer);
fclose($fp);

exec("cd ".$tmp_dir."; ".$postmap." ".$tmp_file);

# Rotate our backup files and reload postfix
exec("mv ".$file_path."/virtual.db_backup3 ".$file_path."/virtual.db_backup4");
exec("mv ".$file_path."/virtual.db_backup2 ".$file_path."/virtual.db_backup3");
exec("mv ".$file_path."/virtual.db_backup1 ".$file_path."/virtual.db_backup2");
exec("mv ".$file_path."/virtual.db ".$file_path."/virtual.db_backup1");

exec("mv ".$tmp_dir."/virtual.db ".$file_path."/virtual.db");


exec("/etc/init.d/postfix reload");

if ($mail_report == TRUE) mail($report_email_address,"Plesk Postfix workaround - rebuilding $file_path/virtual.db",$body);
}

}
?>

Add it to your root crontab to run every hour:

0 * * * * /usr/local/plesk9_workarounds/fixPostfix.sh >/dev/null 2>&1



Thats it!, test, test and test again before turning off the test_mode flag.

Best of luck.
 
Thanks a lot!

As I wrote in another thread over here I had no Problem with reserved system aliases, but with with the infunctional reject mechanism of Plesk-9 / Postfix combo.

Based on jandrews great script i made a complete new virtual.db and now every mail to nonexistent mailboxes get rejected with error 550.

If someone is interested here is jandrews modified script, I used for this task. Basically it is his script - so all credits go to jandrews!

#!/usr/bin/php5 -q
<?php

# Config #####################################

$TEST_MODE = TRUE; # Wont write any config files, just echo output to stdout.

$db_host = "localhost";
$db_login = "admin";
$db_password = ""; #leave it empty to use plesk-9 password
$db_name = "psa";

$postmap = "/usr/sbin/postmap";

$file_path = "/var/spool/postfix/plesk";
$tmp_dir = "/tmp";

$mail_report = TRUE;
$report_email_address = "XXXXXXXXXXXXX"; #used as Catch-All of the Serverdomain

##############################################

if ($db_password == "") {
exec("cat /etc/psa/.psa.shadow",$ret);
$db_password = $ret[0];
}


$db = mysql_pconnect($db_host,$db_login,$db_password);
mysql_select_db($db_name,$db);

$body .= "#################\nDebug Output\n#################\n".$output_buffer;
$output_buffer = "";
$cntrow = 0;


### Get Data

$result = mysql_query("
select concat(m.mail_name,'@',d.name) as line0, IF(m.redir_addr<>'',m.redir_addr,concat(m.mail_name,'@',d.name)) line1, ('Mailbox@Domain -> Mailbox@Domain') as type
from mail m left outer join domains d on m.dom_id = d.id
UNION
select concat(a.alias,'@',d.name) as line0, concat(m.mail_name,'@',d.name) as line1, ('Alias@Domain -> Mailbox@Domain') as type
from mail_aliases as a left join mail as m on a.mn_id = m.id
left join domains as d on m.dom_id = d.id
UNION
select concat(m.mail_name,'@',da.name) as line0, IF(m.redir_addr<>'',m.redir_addr,concat(m.mail_name,'@',d.name)) line1, ('Mailbox@Alias-Domain -> Mailbox@Domain') as type
from mail m left outer join domains d on m.dom_id = d.id left join domainaliases da on m.dom_id = da.dom_id
where upper(da.mail) = 'TRUE'
UNION
select concat(a.alias,'@',da.name) as line0, concat(m.mail_name,'@',d.name) as line1, ('Alias@Domain-Alias -> Mailbox@Domain') as type
from mail_aliases as a left join mail as m on a.mn_id = m.id
left join domains as d on m.dom_id = d.id left outer join domainaliases da on m.dom_id = da.dom_id
where upper(da.mail) = 'TRUE'
UNION
select da.name as line0, '1' as line1, ('Domain-Alias') as type
from domainaliases da
where upper(da.mail) = 'TRUE'
UNION
select misc.val as line0, '1' as line1, ('Domain-Alias -> FullHostName') as type
from misc
where upper(param) = 'FULLHOSTNAME'
UNION
select concat('@',misc.val) as line0, '".$report_email_address."' as line1, ('CatchAll@FullHostName') as type
from misc
where upper(param) = 'FULLHOSTNAME'
ORDER BY line1, line0
");

while ($row = mysql_fetch_assoc($result)){

$cntrow = $cntrow + 1;
$line[0] = $row["line0"];
$line[1] = $row["line1"];

$output_buffer .= $line[0]."\t".$line[1]."\n";
$body .= $cntrow.": [".$line[0]." ".$line[1]."] ".$row["type"]."\n";

}

$body .= "\n\nRows processed: ".$cntrow." of ".mysql_num_rows($result)."\n";
mysql_close($db);


if ($TEST_MODE == TRUE){

$body .= "Test-Mode: ON\n";
$body .= "\n\n#################\nPostfix virtual.db Output\n#################\n".$output_buffer;
echo $body;

if ($mail_report == TRUE) mail($report_email_address,"##TEST## Plesk Postfix workaround - rebuilding $file_path/virtual.db",$body);

$tmp_file = $tmp_dir."/virtual";
exec("rm ".$tmp_file);
$fp = fopen($tmp_file,"w");
fwrite($fp,$output_buffer);
fclose($fp);
exec("cd ".$tmp_dir."; ".$postmap." ".$tmp_file);

} else {

$body .= "Test-Mode: OFF\nUpdating virtual.db ...\n";
$tmp_file = $tmp_dir."/virtual";
exec("rm ".$tmp_file);
$fp = fopen($tmp_file,"w");
fwrite($fp,$output_buffer);
fclose($fp);

exec("cd ".$tmp_dir."; ".$postmap." ".$tmp_file);

# Rotate our backup files and reload postfix
exec("mv ".$file_path."/virtual.db_backup3 ".$file_path."/virtual.db_backup4");
exec("mv ".$file_path."/virtual.db_backup2 ".$file_path."/virtual.db_backup3");
exec("mv ".$file_path."/virtual.db_backup1 ".$file_path."/virtual.db_backup2");
exec("mv ".$file_path."/virtual.db ".$file_path."/virtual.db_backup1");

exec("mv ".$tmp_dir."/virtual.db ".$file_path."/virtual.db");


exec("/etc/init.d/postfix reload");

if ($mail_report == TRUE) mail($report_email_address,"Plesk Postfix workaround - rebuilding $file_path/virtual.db",$body);

}
?>

No Mailman Addresses will be created!!
 
@jandrews

Thanks for the script it has been very useful here! I had to correct the compare however (in the MX cleanup loop). Sometimes, domain names are partly in uppercase which cause problem in the compare so I have change it like this:

if (strtolower($row["mx_record"]) == strtolower($our_mx_server.".")) $local_mx = TRUE;


However, I still do not understand the purpose of that check. Why not using all the MX records in the database directly and not do that cleanup step? What is it required for?


Thanks
 
Is this issue fixed in the new 9.0.1 release? Has anyone verified this? Could someone from Parallels comment on this?
 
Is this issue fixed in the new 9.0.1 release? Has anyone verified this? Could someone from Parallels comment on this?

i didnt testet it yet i will wait if other issues come out with this update.... however for me it is not clear from the announcement http://forum.parallels.com/announcement.php?f=465 if it is fixed or not even the last point is not realy clear
11.[*] Bugfixes and improvements - Multiple other bugfixes and improvements were made.

I would like to see an official Bugtracker and what has soved and what not, thats how a comercial Produkt should work

Brujo
 
Yes, the issue is fixed in 9.0.1.

The issue not have been fixed! I have the problem since upgrade plesk 9.0.0 to 9.0.1 :

Feb 2 09:42:52 postfix/local[21996]: 2F8B03F80B5: to=<[email protected]>, orig_to=<info@real_domain_name.com>, relay=local, delay=0, st

:-(

The only issue is to delete the info mailbox and created it again..
Please, how can we solve the problem without delete and create again the mailbox??
 
I am still having the issue as well and updating to 9.0.1 has not fixed the issue. Anyone opened a support ticket for this issue and had it repaired sucessfully?
 
Gentlemen,

We need more details about how to the issue could be reproduced.
The words like "it still does not work" are useless and don't allow us to understand what does exactly not work.

Thank you for the cooperation.
 
If one wants to screw up a virtual.db file - the easiest way is to run /usr/local/psa/admin/sbin/mchk, which completely rebuilds all the databases.

setup a couple test domains with addresses like user1, info, postmaster, user2, etc.

Now the problem is that in virtual.db generation, addreses
 
The easiest way to screw up virtual.db is to rerun /usr/local/psa/admin/sbin/mchk, which rebuilds databases.
For some reasons, these addresses are hardcoded and are not respected. For every domain one gets these aliases:

[email protected]ldomain
[email protected]ldomain
[email protected]ldomain
[email protected]ldomain
[email protected]ldomain
[email protected]ldomain
[email protected]ldomain

so these addresses (info, anonymous, root, support and the rest) are unusable, mchk does not respect them.
 
I know once i updated to 9.0 that i started to have the issues with info@ ( probably the most common email that is affected ) I went and deleted and readded a few of the email addresses to fix it which it did. Then i ran the update 9.0.2 and all the email addresses reverted back to not working. I since fixed one of them and ran /usr/local/psa/admin/sbin/mchk as ramuva said and now that email is no longer functioning. In my opinion this is probably the worst bug I ever have had with Plesk. I can confirm that I am having the issue on all 3 of my Plesk servers. All servers are running RHEL 5.
 
I know once i updated to 9.0 that i started to have the issues with info@ ( probably the most common email that is affected ) I went and deleted and readded a few of the email addresses to fix it which it did. Then i ran the update 9.0.2 and all the email addresses reverted back to not working. I since fixed one of them and ran /usr/local/psa/admin/sbin/mchk as ramuva said and now that email is no longer functioning. In my opinion this is probably the worst bug I ever have had with Plesk. I can confirm that I am having the issue on all 3 of my Plesk servers. All servers are running RHEL 5.

Onsightdata, just a quick note to say, that would have been 9.0.0.2, 9.0.2 hasn't been released yet, the most recent version is 9.0.1

BTW The info@ bug was one of the few that I didn't encounter, perhaps because I did a backup and fresh install rather than a 'simple' update (updating wouldn't work for me - it failed on many counts).
 
Gentlemen,

We need more details about how to the issue could be reproduced.
The words like "it still does not work" are useless and don't allow us to understand what does exactly not work.

Thank you for the cooperation.

You can't be serious.. you have more than enough information to fix this. It's a recurrent bug that has been present since 9.0.0 which way too many people complained about already. I can't even believe it actually happend AGAIN after I upgraded from 9.0.0 to 9.0.1.
 
Today I did a yum update on one of my servers and a bunch of new psa-postfix and qmail files where available for download. I use the atomic repo. Well since updating that it resolved the issue and now info@ and the other various email addresses work. Thats all the info i have at the moment will update in a bit
 
Gentlemen,

I was actually wrong, sorry. The issue with the aliases after upgrade on postfix still exists.

Thank you for the report. The issue will be fixed as soon as possible. At least since the next update.
 
This is not an acceptable timeframe, it's been over 2 months since the release of Plesk 9.
 
The easiest way to screw up virtual.db is to rerun /usr/local/psa/admin/sbin/mchk, which rebuilds databases.
For some reasons, these addresses are hardcoded and are not respected. For every domain one gets these aliases:

[email protected]ldomain
[email protected]ldomain
[email protected]ldomain
[email protected]ldomain
[email protected]ldomain
[email protected]ldomain
[email protected]ldomain

so these addresses (info, anonymous, root, support and the rest) are unusable, mchk does not respect them.

Absolutely right.
The root of problem is in mchk utility which is setting up the aliases by mistake (for postfix MTA only).
mchk is called automatically during upgrade and breaks aliases.

If you use postfix mta:
To fix the problem you can try to apply the following steps to your 9.0.1 installation:
1) download mailsrv_entities_dump utility for your OS and arch from ftp://download1.parallels.net/Plesk/Hotfix/PleskUnix/9.0.1/149248/
2) put it into the /usr/lib/plesk-9.0/ directory
3) run /usr/local/psa/admin/sbin/mchk

mailsrv_entities_dump is a part of mchk which is responsible for mail aliases.
 
Back
Top