• Please be aware: Kaspersky Anti-Virus has been deprecated
    With the upgrade to Plesk Obsidian 18.0.64, "Kaspersky Anti-Virus for Servers" will be automatically removed from the servers it is installed on. We recommend that you migrate to Sophos Anti-Virus for Servers.
  • The Horde webmail has been deprecated. Its complete removal is scheduled for April 2025. For details and recommended actions, see the Feature and Deprecation Plan.
  • We’re working on enhancing the Monitoring feature in Plesk, and we could really use your expertise! If you’re open to sharing your experiences with server and website monitoring or providing feedback, we’d love to have a one-hour online meeting with you.

"Domain Name Not Found" - Googlebot

G

gckorn

Guest
We upgraded to 8.6 in mid-August. Since that time, and with each Googlebot crawl, an ever increasing number of our pages are being dropped from the Google index.

Google Webmaster Tools (GWT) reports an HTTP access error of "Domain Name Not Found." The definition of this error is as follows: "The DNS may not have been properly resolving. We could communicate with the DNS server, but couldn't find the domain name." Example domain experiencing this issue - though it is common to all domains on the server - is w w w . m a i n e . i n f o

GWT also shows a graph demonstrating that "Time spent downloading a page (in milliseconds)" has increased after the Plesk 8.6 install from an average of 300 ms to over 800 ms and has stayed constant at that level.

The host company that runs the server says that everything is fine. But they can see little or no Googlebot activity in the access logs. This suggests to me that Googlebot is - as it reports - not getting past the DNS server.

The host company suggested that I use an A record for the www subdomain rather than CNAME. I have tried this but do not know the results yet. However, in doing so, I found that certain exclusive IP addresses are not resolving as they should. I am told this is an Apache problem, but I figure it may preclude the A record approach anyway.

Also during troubleshooting I discovered the SOA serial numbers for most domains were now set to zero. At the request of the hosting company, I went through 400+ domains on the Plesk CP and checked the "DNS zone serial number format" box (it had been unchecked for most domains ... and a script fix did not cause checking, so had to do it manually over several hours). Now all domains have the same, identical serial number (today's date). I am told by the hosting company that this is okay. Uncertain if it will help the Googlebot problem.

The host company also suggested that I have Google send me a traceroute from the googlebot to my dedicated server. Common sense instructs that that won't happen.

I know that Plesk 8.6 made some changes to DNS. But I don't know if any caused this impact. Something clearly did, as we were getting along just fine before 8.6.

The bottom line is that Google is the lifeblood of my websites, and of other clients who are on the server. As it is for many. If a Plesk 8.6 DNS-related issue is blocking the googlebot - and GWT states that it can communicate with the DNS server but cannot then find the domain name - then this is a severe problem. Doing nothing is not an alternative.

The hosting company is shrugging shoulders on this. Would certainly appreciate some ideas and assistance.

gckorn
 
Using A records for @ and www are probably a good thing, using a cname of @ can actually break something like email in some cases becuase some bad servers out there do not parse the MX record (which is also the @) if there is a fqdn in the @ record.

can you replicate the problem google reports?
IE if yuo do an nslookup or some name resoltuion command against your name servers do you get the NX response ?

i agree that expecting google to do an nslookup or traceroute is probably not going to occur, but its worth a shot. It would at least indicate if they are going to the right place to get your info.

Also make sure you dont have a firewall or anything that may be blocking some of the queries
 
Please click one of the Quick Reply icons in the posts above to activate Quick Reply.
 
A records

Thanks Hosting Guy. I agree with A records and am tempted to acquire exclusive IPs for all of my domains. I finally have looked at the access logs and see plenty of googlebot activity. No googlebot entries in the error logs.

gckorn
 
Back
Top