• If you are still using CentOS 7.9, it's time to convert to Alma 8 with the free centos2alma tool by Plesk or Plesk Migrator. Please let us know your experiences or concerns in this thread:
    CentOS2Alma discussion

search engines getting a 404 when trying to crawl domain

B

butler

Guest
Hi

When I submit all the domains on my server to the search engines, they are getting a 404 return.

however if I submit using the ip, they pull up index page with no problem

it's like something is blocking the search engine crawler,

I can pull the domain up in a browser window by typing in the url with no problem

I have talked to the tech's at the company where my deicated server is, but they dismiss me when I say the word search engine. All they can think is I want them to tell me how to get a listing.

I have tried to explain it, but it's like going against a wall.

Please help, or if you need more information, please let me know

Wlee
 
It sounds like your search engine does not support HTTP1.1 (i.e. does not send Host: header with the request). All major search engines do have such support. Which one are you trying to use? Check their FAQ or ask their support if they do.

If they don't, all you can do is to use one dedicated IP per each your site.
 
Yes, additional JavaScript may fool some search engines too. You should disable @domains feature by unchecking "enable [email protected] access format" checkbox then.
 
Back
Top