Google is blocking my mail server for some unknown reason

[Matt Grant]

My mail server running Plesk 12.0.18 has been blocked by Google (Gmail and Google Business Apps) for over a week now. I have filled out the unblocking form daily and nothing is happening. I get no response when I fill out the form and I have no idea why I am being blocked. I don't understand why Google cannot send me a warning to give me a chance to crack down on whoever caused this to happen.

My suspicion is that my client's send legitimate email to Google hosted accounts on a daily basis and that is causing the block to last longer than it should (blockage usually lasts 3-5 days from my research). I have gone over the server with a fine tooth comb and all security updates are current. i also have rate limiters on all of my domains. I have poured over the server's logs and do not see any type of SPAM coming from the server. MxToolBox shows that I am not on any of the blacklists and that all of my DNS is setup and working correctly (the server has been up for a year or so) and my server's Sender Score is 99.

Is there a temporary way to block email from even being attempted to be sent to Google accounts? I just want to stop anything from being sent to their domains for a few days to see if that gets me unblocked.

My client's are freaking out and I do not know what to do otherwise.

Can anyone help me?

 

[J-F Brouillette]

Hi Matt,

Could you provide log from syslog (the 550 error message from Google)

Thanks

 

[Matt Grant]

Please forgive my ignorance, but where can I find the syslog? I googled it and it says it should be in /var/log/syslog but it is not there.

Here is the bounce message I get from Google, if that helps...

Hi. This is the qmail-send program at mail.digitaltelemetry.com.
I'm afraid I wasn't able to deliver your message to the following addresses.
This is a permanent error; I've given up. Sorry it didn't work out.

<[email protected]>:
2607:xxxx:xxxx:xxxx:0000:0000:0000:001a failed after I sent the message.
Remote host said: 550-5.7.1 [2607:xxxx:xxx:xxxx::xx:xxxx 12] Our system has detected that
550-5.7.1 this message is likely unsolicited mail. To reduce the amount of spam
550-5.7.1 sent to Gmail, this message has been blocked. Please visit
550 5.7.1 https://support.google.com/mail/answer/188131 for more information. v6si7190715igz.77 - gsmtp

This is part of what I have been posting to the Google unblock form from my maillog

Feb 29 11:17:28 mail qmail: 1456762648.612725 delivery 8924: failure: 2607:xxxx:xxxx:xxxx:0000:0000:0000:001b_failed_after_I_sent_the_message./Remote_host_said:_550-5.7.1_[2607:xxxx:xxx:xxxx::xx:xxxx______12]_Our_system_has_detected_tha$


Thanks for the fast reply, hopefully you can help me figure this out.

Hi Matt,

Could you provide log from syslog (the 550 error message from Google)

Thanks

 

[J-F Brouillette]

I had problems with a VPS that had an IPv6 in the past (sending email to google users)
Check this out: http://tanguy.ortolo.eu/blog/article109/google-ipv6-smtp-restrictions maybe it will help!

 

[Matt Grant]

Thanks for the link, but the error is not about IPV6 and I am not running Postfix. Any other tips?

I had problems with a VPS that had an IPv6 in the past (sending email to google users)
Check this out: http://tanguy.ortolo.eu/blog/article109/google-ipv6-smtp-restrictions maybe it will help!

 

[J-F Brouillette]

Which system are you using for SMTP then?

Can you test this: telnet myip.gelma.net (it will show you if you have a public ipv6)

I don't know yourt echnical skill, don't want to push it but sometimes your provider exit with an ipv6 and you dont know about it.

 

[Matt Grant]

I am using Qmail for SMTP.

Here are the results from the telnet session you referenced...

[root@mail log]# telnet myip.gelma.net
Trying 2600:3c03::f03c:91ff:fe96:cc28...
Connected to myip.gelma.net.
Escape character is '^]'.



Your IPv4: 9exx




Connection closed by foreign host.


I know enough to be dangerous on Linux, but not enough to be a pro. Why would this all of a sudden happen if it was caused by my IPV6 address?

Which system are you using for SMTP then?

Can you test this: telnet myip.gelma.net (it will show you if you have a public ipv6)

I don't know yourt echnical skill, don't want to push it but sometimes your provider exit with an ipv6 and you dont know about it.

 

[J-F Brouillette]

I am using Qmail for SMTP.

Here are the results from the telnet session you referenced...

[root@mail log]# telnet myip.gelma.net
Trying 2600:3c03::f03c:91ff:fe96:cc28...
Connected to myip.gelma.net.
Escape character is '^]'.

Clearly your server is trying to contact with ipv6 "Trying 2600:3c03::f03c:91ff:fe96:cc28..."
For example, I got an IPv6 from my ISP since 2 months, didnt get any notification.

If its not your case can't help more sorry. We don't have any control over how google classify email :\

 

[trialotto]

@Matt Grant

It certainly is not about IPv6, Google accepts "that kind" of mail normally (and I certainly would not base conclusions on non-Google articles from 2013).

In essence, Google does block a lot of mail, even though there is not a reason to block the mail: the "machine is continuously learning", but it often is not that smart.

The best methods to get mail not ending up in the spam folders of Gmail are:

a) DKIM: go to "Tools & Settings > Mail > Mail Server Settings (click on it) > DomainKeys spam protection": enable sign outgoing mail

b) use an appropriate SPF record: add a TXT record to DNS (Note: Google has some simple documentation about this)

c) use DMARC: add a TXT record to DNS

I have to mention that option a) is the most convenient option, options b) and c) are more advanced.

I also have to admit that, in the case of Gmail, there is no absolute guarantee that mails does not end up in the Spam folder: the learning curve of the "machine" can be quite steep, in the sense that many mails will be viewed as spam, as long as the users of Gmail do not declare that the mail in the Spam folder is actually not spam at all.

A small tip: you can help the "machine" learning faster, by sending yourself some mails and accept them as "not being spam", if they end up in the spam folder.

Another small hint: Google also blocks specific mails based upon the origin of the mail, in the sense that mail servers with a bad reputation are automatically blocked.

A small tip: check on senderbase.org whether your mail server has a bad reputation and/or check on DNSBL blacklists whether the IP of your server or some domains have been blocked.

Hope the above helps!

Regards....

 

[Matt Grant]

I appreciate your lengthy response, but my issue is not that emails sent from my server are going to the SPAM folders of Google hosted email accounts. The issue is that Google is flat out blocking my server from sending email to their servers. If anyone on my server emails a Google hosted account it bounces back instantly and I also cannot telnet to any of their mail servers. For some unknown reason, they are blocking my server and I have filled out the de-list/unblock form many times over the last 8 days and it is still blocked. My server is not listed on any of the 208 public blacklists on MxToolBox, my sender score is 99 on Sender Score and senderbase.org says my server's email reputation is good. I am at a loss as to why I am blocked and have done everything Google has asked me to do as far as filling out the form. I am almost ready to just order a new server from 1&1, but I am concerned that as soon as I get it setup, it will be blocked too.

I really wish Google would contact the owners of server's to warn them of potential blocks, to give them a chance to stop whatever is causing them to feel the need to block a server before they block a server. I run a pretty tight ship on my servers and would crack down hard on a user/domain that was causing this to happen. If I could only block my server from anyone sending email to any Google hosted accounts for a few days, I bet the block would be lifted. But alas no one seems to know how to implement this. FML!!!!

@Matt Grant

It certainly is not about IPv6, Google accepts "that kind" of mail normally (and I certainly would not base conclusions on non-Google articles from 2013).

In essence, Google does block a lot of mail, even though there is not a reason to block the mail: the "machine is continuously learning", but it often is not that smart.

The best methods to get mail not ending up in the spam folders of Gmail are:

a) DKIM: go to "Tools & Settings > Mail > Mail Server Settings (click on it) > DomainKeys spam protection": enable sign outgoing mail

b) use an appropriate SPF record: add a TXT record to DNS (Note: Google has some simple documentation about this)

c) use DMARC: add a TXT record to DNS

I have to mention that option a) is the most convenient option, options b) and c) are more advanced.

I also have to admit that, in the case of Gmail, there is no absolute guarantee that mails does not end up in the Spam folder: the learning curve of the "machine" can be quite steep, in the sense that many mails will be viewed as spam, as long as the users of Gmail do not declare that the mail in the Spam folder is actually not spam at all.

A small tip: you can help the "machine" learning faster, by sending yourself some mails and accept them as "not being spam", if they end up in the spam folder.

Another small hint: Google also blocks specific mails based upon the origin of the mail, in the sense that mail servers with a bad reputation are automatically blocked.

A small tip: check on senderbase.org whether your mail server has a bad reputation and/or check on DNSBL blacklists whether the IP of your server or some domains have been blocked.

Hope the above helps!

Regards....

 

[trialotto]

@Matt Grant

To be honest, ordering a new server from 1and1 is not a good plan, this is probably one of the most important factors in your current issue.

In the forum, many mail related issues can be found with respect to 1and1 servers, mostly due to an improper setting of "mail and server infrastructure".

The essence of all those issues is that resolving DNS (read: hostnames) is a little bit impossible. That is the rather simplified summary.

In your case, that would imply that Google automatically blocks mail from any of your mail servers, hosted on 1and1 machines (and it is very likely that something has occurred that can imply that Google currently blocks all IPs of 1and1 servers).

In short, I would strongly recommend to set up a Plesk instance at a different hosting provider (otherwise, your current issue would be reoccurring in due time).

Regards....

 

[Matt Grant]

I have purchased many dedicated servers from 1&1 for well over 10 years and besides a couple of hardware failures 7 or 8 years ago, I have had no issues with them as a server provider whatsoever. Their support is top notch and their server pricing really cannot be beat. Any server provider could potentially be blacklisted, no matter who they are. If the block was because of 1&1, I would have to imagine 1000's of people would be flooding their support forums as well as any server support forum with complaints about Google blocking their servers. I am content with 1&1, I just need to figure out how to get Google unblock my server.

I do appreciate your informative insights and help with my issue, but leaving 1&1 is not something I am considering at this time.

@Matt Grant

To be honest, ordering a new server from 1and1 is not a good plan, this is probably one of the most important factors in your current issue.

In the forum, many mail related issues can be found with respect to 1and1 servers, mostly due to an improper setting of "mail and server infrastructure".

The essence of all those issues is that resolving DNS (read: hostnames) is a little bit impossible. That is the rather simplified summary.

In your case, that would imply that Google automatically blocks mail from any of your mail servers, hosted on 1and1 machines (and it is very likely that something has occurred that can imply that Google currently blocks all IPs of 1and1 servers).

In short, I would strongly recommend to set up a Plesk instance at a different hosting provider (otherwise, your current issue would be reoccurring in due time).

Regards....

 

[trialotto]

@Matt Grant

You stated

If the block was because of 1&1, I would have to imagine 1000's of people would be flooding their support forums as well as any server support forum with complaints about Google blocking their servers.

and that is actually the case.

Have a look at the Plesk forum, some examples do exist that involve 1and1 servers.

Naturally, you are right to say that one should stick to a trusted provider and/or a provider that serves you to satisfaction.

However, you are not right to say that any other hosting provider can get blocked (for instance, on black- and blocklists): it is not the provider that gets blocked, it is the IP.

As a result, one can safely conclude that mismanagement of any server or mail server will increase the probability on blockage of the server.

The strange situation with 1and1 is that the nature of their infrastructure has been and still is increasing the before mentioned probability even more.

Note that 1and1 has acknowledged the fact that their servers in particular get blocked by many third parties.

Also note that 1and1 has released some documentation with respect to resolving mail issues, with that documentation essentially stating that one has to define a PTR record explicitly.

If you ask me, the latter documentation is a little bit strange: it is the task of the provider that all PTR records are set properly and are aligned with international standards.

In short, you can resolve your issues by following the 1and1 documentation (i.e. set PTR records) AND/OR by following the solution I mailed you.

However, be aware of the fact that it can take a considerable amount of time before Google unblocks you (i.e. as result of the "machine learning curve").

Regards.....

 

[Matt Grant]

It is strange I just searched the Plesk forums as well as the other forums and I do not see anyone with any complaints about Google blocking their server, let alone blocking 1&1 in the last few weeks (let alone this year).

https://talk.plesk.com/search/576980/?q=google&o=date&c[title_only]=1&c[node]=735
https://talk.plesk.com/search/576982/?q=gmail&o=date&c[title_only]=1&c[node]=735

https://www.google.com/?gws_rd=ssl#q=gmail+blocking+my+email+server
https://www.google.com/?gws_rd=ssl#q=google+blocking+my+email+server

I understand you have some sort of issue with 1&1, but I am not sure if you have ever used them for a dedicated server or are just going by complaints on this forum from people who have no business managing a dedicated server and have them mis-configured. This server is almost 2 years old and the IP address has been the same since I got it. This same block happened on my last server about 2.5 years ago, but it was caused by an client's computer that was infected with some sort of spamming virus and it blasted email out for almost a week. The computer was reloaded and I filled out the form to get it unblocked and 3 days later it was. Since then, there have been no issues. I run a pretty tight ship when it comes to my servers, I check them weekly for updates and check my Magic Spam logs (it has an awesome searchable log feature on it's control panel and it shows me all email sent/received) almost daily to make sure nothing is going out that shouldn't be. My PTR record is setup correctly and my DNS resolves properly. I will admit, I tried to setup a DMARC record yesterday on my registrar's DNS control panel (I do not run DNS on my Plesk server, I use my registrar for that) and for some reason when I run a test on it, it says I do not have one setup. I plan on getting that figured out soon. My SPF record checks out fine and I am working towards implementing Domain Keys/DKIM soon. All of that is besides the point, since the bounce message I am getting is "550-5.7.1 this message is likely unsolicited mail. To reduce the amount of spam sent to Gmail, this message has been blocked. Please visit https://support.google.com/mail/answer/188131 for more information. u67si12506178ioi.61 - gsmtp". There is a different bounce message error if your PTR or DNS is not setup correctly. My customers and myself can email every other domain (besides Gmail and Google hosted domains) and not get bounces. This issue is only with Google's mail servers.

Again I truly appreciate the thorough information you are giving me and wish what you were telling me applied to my issue.

@Matt Grant

You stated



and that is actually the case.

Have a look at the Plesk forum, some examples do exist that involve 1and1 servers.

Naturally, you are right to say that one should stick to a trusted provider and/or a provider that serves you to satisfaction.

However, you are not right to say that any other hosting provider can get blocked (for instance, on black- and blocklists): it is not the provider that gets blocked, it is the IP.

As a result, one can safely conclude that mismanagement of any server or mail server will increase the probability on blockage of the server.

The strange situation with 1and1 is that the nature of their infrastructure has been and still is increasing the before mentioned probability even more.

Note that 1and1 has acknowledged the fact that their servers in particular get blocked by many third parties.

Also note that 1and1 has released some documentation with respect to resolving mail issues, with that documentation essentially stating that one has to define a PTR record explicitly.

If you ask me, the latter documentation is a little bit strange: it is the task of the provider that all PTR records are set properly and are aligned with international standards.

In short, you can resolve your issues by following the 1and1 documentation (i.e. set PTR records) AND/OR by following the solution I mailed you.

However, be aware of the fact that it can take a considerable amount of time before Google unblocks you (i.e. as result of the "machine learning curve").

Regards.....

 

[trialotto]

@Matt Grant

I am surprised: you said that you already had a hack, a block due to some infected account, a whole week of spamming.......and so on.

Furthermore, I am surprised about this bit:

My PTR record is setup correctly and my DNS resolves properly. I will admit, I tried to setup a DMARC record yesterday on my registrar's DNS control panel (I do not run DNS on my Plesk server, I use my registrar for that) and for some reason when I run a test on it, it says I do not have one setup. I plan on getting that figured out soon.

The surprise is simply the endresult of what you say between the lines: a spam attack during one week is not "good practice" for a sysadmin, while not being able to set a DMARC record (i.e. a quite simple DNS TXT record) is something related to 1and1 infrastructure.

I really have no beef with 1and1, but I am rather flabergasted about their customer service: all issues we talk about, are known for years to 1and1 (and not resolved).

It is in the best interest of forum members to know what they can expect with 1and1, as such the only reason why I mention specific details about 1and1.

These details are based upon a continuous test run of multiple servers at 1and1, for a long time period (i.e. more than 2 years).

Naturally, forum members are entitled to make their own choices.

That is all, there is only the intention to provide some relevant information to forum members.

Yes, this relevant information also concerns Google, that really applies a "strange" learning curve to "spam learning", with a lot of forum members pulling out their hairs, due to mail blocks.

Regards......

 

[trialotto]

By the way, speaking of which, Google also puts mails about this specific topic thread in the spam folder, while all other Plesk forum mails are in the inbox.

Grinn, a bad omen?

 

[Matt Grant]

I never said I was hacked. What I said was, one of my client's (that I host their company website and email) home computer (one that I have never touched) got a virus on it and it sent out emails for around a week. This was on my old mail server with a different IP address over 2 years ago running Plesk 11. It just so happened on a week where I was on vacation and on Plesk 11 there is not outgoing mail rate limiter like on Plesk 12 and is why I was not notified about it.

As for the DMARC problem, I generated a DMARC entry on the unlocktheinbox.com website. I then added it to my TXT record on my registrar's DNS control panel:

v=DMARC1; p=reject; sp=none; [email protected]; [email protected]; rf=afrf; pct=100; ri=86400 (I x'd out my email address to keep it hidden on the forum)

This is the transcript of the DMARC test on MxToolBox

No DMARC Records exist<br /> PWS3v2 203ms

0 k.gtld-servers.net 192.52.178.30 NON-AUTH 156 ms Received 2 Referrals , rcode=NO_ERROR digitaltelemetry.com. 172800 IN NS ns1.dotster.com,digitaltelemetry.com. 172800 IN NS ns2.dotster.com,

1 ns1.dotster.com 66.96.142.146 AUTH 46 ms Received 1 Referrals , rcode=NAME_ERROR digitaltelemetry.com. 3600 IN SOA mname=ns1.yourhostingaccount.com rname=admin.yourhostingaccount.com serial=2012112682,

I use a Gmail account for my hostmaster related emails, so in the case there is a server outage or problem with the server, emails will still get through to me. I have been hosting websites/email since 1999 and pretty well versed on the ins and outs of hosting. I will admit, some of the newer SPAM prevention measures like SPF, DKIM and DMARC are new to me and I am trying to get up to speed on them.

@Matt Grant

I am surprised: you said that you already had a hack, a block due to some infected account, a whole week of spamming.......and so on.

Furthermore, I am surprised about this bit:



The surprise is simply the endresult of what you say between the lines: a spam attack during one week is not "good practice" for a sysadmin, while not being able to set a DMARC record (i.e. a quite simple DNS TXT record) is something related to 1and1 infrastructure.

I really have no beef with 1and1, but I am rather flabergasted about their customer service: all issues we talk about, are known for years to 1and1 (and not resolved).

It is in the best interest of forum members to know what they can expect with 1and1, as such the only reason why I mention specific details about 1and1.

These details are based upon a continuous test run of multiple servers at 1and1, for a long time period (i.e. more than 2 years).

Naturally, forum members are entitled to make their own choices.

That is all, there is only the intention to provide some relevant information to forum members.

Yes, this relevant information also concerns Google, that really applies a "strange" learning curve to "spam learning", with a lot of forum members pulling out their hairs, due to mail blocks.

Regards......

 

[trialotto]

@Matt Grant

Alter the DMARC record, there is a ";" at the end of the record that should not be there.

If I am not mistaken, that should be it (even though for DMARC records the ";" can do no harm, it can affect DNS systems of providers, in the sense that they do not recognize it).

However, the DMARC record for unlocktheinbox.com is completely different from the text you posted in your last response. How come?

Regards

 

[Matt Grant]

I am not sure why. My registrar Dotster, when I tried to add the DMARC record as TXT it was not savign properly on my control panel. I had to call their support to add it for me and that is what they added. Are you talking about the ; after v=DMARC1; ?

I just ran the wizard again and this is what it gave me:

"v=DMARC1; p=reject; sp=none; rua=mailto:[email protected]; ruf=mailto:[email protected]; rf=afrf; pct=100; ri=86400"

This is what shows on my DNS control panel:

v=DMARC1; p=reject; sp=none; [email protected]; [email protected]; rf=afrf; pct=100; ri=86400

It looks like they did not add the mailtto: section. I will have to call them again to get it corrected. Thanks for the keen eye!

@Matt Grant

Alter the DMARC record, there is a ";" at the end of the record that should not be there.

If I am not mistaken, that should be it (even though for DMARC records the ";" can do no harm, it can affect DNS systems of providers, in the sense that they do not recognize it).

However, the DMARC record for unlocktheinbox.com is completely different from the text you posted in your last response. How come?

Regards

 

[trialotto]

@Matt Grant

This is the current DMARC on the unlocktheinbox.com site:

v=DMARC1;p=none;pct=100;rua=mailto:[email protected];ruf=mailto:[email protected];

Note the last ";" (and also note that I cleaned up the line a bit, in DNS propagation, this line has some backslashes in it).

Regards....