Question Fail2Ban issue with Google Sitemaps

[Chagui900]

Server operating system version

CentOS Linux 7.9.2009 (Core)

Plesk version and microupdate number

Plesk Obsidian v18.0.62_build1800240724.11 os_CentOS 7

Fail2Ban bloquea mi Googlebot y me da un error al leer los Sitemaps alojados en la consola de búsqueda. ¿Alguien me puede ayudar?

[curtesy admin translation]

Fail2Ban crashes my Googlebot and gives me an error when reading the Sitemaps hosted in the search console. Can anyone help me?

 

[Sebahat.hadzhi]

If you haven't already, please add Googlebot's IP addresses to the Fail2Ban's Whitelist. I hope that helps.

 

[pleskpanel]

If you haven't already, please add Googlebot's IP addresses to the Fail2Ban's Whitelist. I hope that helps.

Since adding these one by one is somewhat cumbersome, it would be great to see a one-click "Add/Update Googlebot known IP addresses" given their importance in the search world but in the meantime, based on the currently published IP list above, would this command work?

# plesk bin ip_ban --add-trusted "2001:4860:4801:10::/64 2001:4860:4801:11::/64 2001:4860:4801:12::/64 2001:4860:4801:13::/64 2001:4860:4801:14::/64 2001:4860:4801:15::/64 2001:4860:4801:16::/64 2001:4860:4801:17::/64 2001:4860:4801:18::/64 2001:4860:4801:19::/64 2001:4860:4801:1a::/64 2001:4860:4801:1b::/64 2001:4860:4801:1c::/64 2001:4860:4801:1d::/64 2001:4860:4801:1e::/64 2001:4860:4801:1f::/64 2001:4860:4801:20::/64 2001:4860:4801:21::/64 2001:4860:4801:22::/64 2001:4860:4801:23::/64 2001:4860:4801:24::/64 2001:4860:4801:25::/64 2001:4860:4801:26::/64 2001:4860:4801:27::/64 2001:4860:4801:28::/64 2001:4860:4801:29::/64 2001:4860:4801:2::/64 2001:4860:4801:2a::/64 2001:4860:4801:2b::/64 2001:4860:4801:2c::/64 2001:4860:4801:2d::/64 2001:4860:4801:2e::/64 2001:4860:4801:2f::/64 2001:4860:4801:31::/64 2001:4860:4801:32::/64 2001:4860:4801:33::/64 2001:4860:4801:34::/64 2001:4860:4801:35::/64 2001:4860:4801:36::/64 2001:4860:4801:37::/64 2001:4860:4801:38::/64 2001:4860:4801:39::/64 2001:4860:4801:3a::/64 2001:4860:4801:3b::/64 2001:4860:4801:3c::/64 2001:4860:4801:3d::/64 2001:4860:4801:3e::/64 2001:4860:4801:40::/64 2001:4860:4801:41::/64 2001:4860:4801:42::/64 2001:4860:4801:43::/64 2001:4860:4801:44::/64 2001:4860:4801:45::/64 2001:4860:4801:46::/64 2001:4860:4801:47::/64 2001:4860:4801:48::/64 2001:4860:4801:49::/64 2001:4860:4801:4a::/64 2001:4860:4801:50::/64 2001:4860:4801:51::/64 2001:4860:4801:53::/64 2001:4860:4801:54::/64 2001:4860:4801:55::/64 2001:4860:4801:60::/64 2001:4860:4801:61::/64 2001:4860:4801:62::/64 2001:4860:4801:63::/64 2001:4860:4801:64::/64 2001:4860:4801:65::/64 2001:4860:4801:66::/64 2001:4860:4801:67::/64 2001:4860:4801:68::/64 2001:4860:4801:69::/64 2001:4860:4801:6a::/64 2001:4860:4801:6b::/64 2001:4860:4801:6c::/64 2001:4860:4801:6d::/64 2001:4860:4801:6e::/64 2001:4860:4801:6f::/64 2001:4860:4801:70::/64 2001:4860:4801:71::/64 2001:4860:4801:72::/64 2001:4860:4801:73::/64 2001:4860:4801:74::/64 2001:4860:4801:75::/64 2001:4860:4801:76::/64 2001:4860:4801:77::/64 2001:4860:4801:78::/64 2001:4860:4801:79::/64 2001:4860:4801:80::/64 2001:4860:4801:81::/64 2001:4860:4801:82::/64 2001:4860:4801:83::/64 2001:4860:4801:84::/64 2001:4860:4801:85::/64 2001:4860:4801:86::/64 2001:4860:4801:87::/64 2001:4860:4801:88::/64 2001:4860:4801:90::/64 2001:4860:4801:91::/64 2001:4860:4801:92::/64 2001:4860:4801:93::/64 2001:4860:4801:c::/64 2001:4860:4801:f::/64 192.178.5.0/27 192.178.6.0/27 34.100.182.96/28 34.101.50.144/28 34.118.254.0/28 34.118.66.0/28 34.126.178.96/28 34.146.150.144/28 34.147.110.144/28 34.151.74.144/28 34.152.50.64/28 34.154.114.144/28 34.155.98.32/28 34.165.18.176/28 34.175.160.64/28 34.176.130.16/28 34.22.85.0/27 34.64.82.64/28 34.65.242.112/28 34.80.50.80/28 34.88.194.0/28 34.89.10.80/28 34.89.198.80/28 34.96.162.48/28 35.247.243.240/28 66.249.64.0/27 66.249.64.128/27 66.249.64.160/27 66.249.64.224/27 66.249.64.32/27 66.249.64.64/27 66.249.64.96/27 66.249.65.0/27 66.249.65.160/27 66.249.65.192/27 66.249.65.224/27 66.249.65.32/27 66.249.65.64/27 66.249.65.96/27 66.249.66.0/27 66.249.66.160/27 66.249.66.192/27 66.249.66.32/27 66.249.66.64/27 66.249.66.96/27 66.249.68.0/27 66.249.68.32/27 66.249.68.64/27 66.249.69.0/27 66.249.69.128/27 66.249.69.160/27 66.249.69.192/27 66.249.69.224/27 66.249.69.32/27 66.249.69.64/27 66.249.69.96/27 66.249.70.0/27 66.249.70.128/27 66.249.70.160/27 66.249.70.192/27 66.249.70.224/27 66.249.70.32/27 66.249.70.64/27 66.249.70.96/27 66.249.71.0/27 66.249.71.128/27 66.249.71.160/27 66.249.71.192/27 66.249.71.224/27 66.249.71.32/27 66.249.71.64/27 66.249.71.96/27 66.249.72.0/27 66.249.72.128/27 66.249.72.160/27 66.249.72.192/27 66.249.72.224/27 66.249.72.32/27 66.249.72.64/27 66.249.72.96/27 66.249.73.0/27 66.249.73.128/27 66.249.73.160/27 66.249.73.192/27 66.249.73.224/27 66.249.73.32/27 66.249.73.64/27 66.249.73.96/27 66.249.74.0/27 66.249.74.128/27 66.249.74.32/27 66.249.74.64/27 66.249.74.96/27 66.249.75.0/27 66.249.75.128/27 66.249.75.160/27 66.249.75.192/27 66.249.75.224/27 66.249.75.32/27 66.249.75.64/27 66.249.75.96/27 66.249.76.0/27 66.249.76.128/27 66.249.76.160/27 66.249.76.192/27 66.249.76.224/27 66.249.76.32/27 66.249.76.64/27 66.249.76.96/27 66.249.77.0/27 66.249.77.128/27 66.249.77.160/27 66.249.77.192/27 66.249.77.224/27 66.249.77.32/27 66.249.77.64/27 66.249.77.96/27 66.249.78.0/27 66.249.78.32/27 66.249.79.0/27 66.249.79.128/27 66.249.79.160/27 66.249.79.192/27 66.249.79.224/27 66.249.79.32/27 66.249.79.64/27 66.249.79.96/27" -description "Googlebot"

 

[Sebahat.hadzhi]

@pleskpanel if you want to specify a description, you need to separate the IP addresses/ranges with ;. For example:

plesk bin ip_ban --add-trusted "2001:4860:4801:10::/64;2001:4860:4801:11::/64;2001:4860:4801:12::/64" -description "Googlebot"

 

[pleskpanel]

@pleskpanel if you want to specify a description, you need to separate the IP addresses/ranges with ;. For example:

The Plesk CLI documentation page doesn't make any reference to the usage of semicolons versus spaces so what does that mean in the context of mixing IPs with CIDR ranges?

For example, does the full Googlebot IP set (with a description) become:

# plesk bin ip_ban --add-trusted "2001:4860:4801:10::/64;2001:4860:4801:11::/64;2001:4860:4801:12::/64;2001:4860:4801:13::/64;2001:4860:4801:14::/64;2001:4860:4801:15::/64;2001:4860:4801:16::/64;2001:4860:4801:17::/64;2001:4860:4801:18::/64;2001:4860:4801:19::/64;2001:4860:4801:1a::/64;2001:4860:4801:1b::/64;2001:4860:4801:1c::/64;2001:4860:4801:1d::/64;2001:4860:4801:1e::/64;2001:4860:4801:1f::/64;2001:4860:4801:20::/64;2001:4860:4801:21::/64;2001:4860:4801:22::/64;2001:4860:4801:23::/64;2001:4860:4801:24::/64;2001:4860:4801:25::/64;2001:4860:4801:26::/64;2001:4860:4801:27::/64;2001:4860:4801:28::/64;2001:4860:4801:29::/64;2001:4860:4801:2::/64;2001:4860:4801:2a::/64;2001:4860:4801:2b::/64;2001:4860:4801:2c::/64;2001:4860:4801:2d::/64;2001:4860:4801:2e::/64;2001:4860:4801:2f::/64;2001:4860:4801:31::/64;2001:4860:4801:32::/64;2001:4860:4801:33::/64;2001:4860:4801:34::/64;2001:4860:4801:35::/64;2001:4860:4801:36::/64;2001:4860:4801:37::/64;2001:4860:4801:38::/64;2001:4860:4801:39::/64;2001:4860:4801:3a::/64;2001:4860:4801:3b::/64;2001:4860:4801:3c::/64;2001:4860:4801:3d::/64;2001:4860:4801:3e::/64;2001:4860:4801:40::/64;2001:4860:4801:41::/64;2001:4860:4801:42::/64;2001:4860:4801:43::/64;2001:4860:4801:44::/64;2001:4860:4801:45::/64;2001:4860:4801:46::/64;2001:4860:4801:47::/64;2001:4860:4801:48::/64;2001:4860:4801:49::/64;2001:4860:4801:4a::/64;2001:4860:4801:50::/64;2001:4860:4801:51::/64;2001:4860:4801:53::/64;2001:4860:4801:54::/64;2001:4860:4801:55::/64;2001:4860:4801:60::/64;2001:4860:4801:61::/64;2001:4860:4801:62::/64;2001:4860:4801:63::/64;2001:4860:4801:64::/64;2001:4860:4801:65::/64;2001:4860:4801:66::/64;2001:4860:4801:67::/64;2001:4860:4801:68::/64;2001:4860:4801:69::/64;2001:4860:4801:6a::/64;2001:4860:4801:6b::/64;2001:4860:4801:6c::/64;2001:4860:4801:6d::/64;2001:4860:4801:6e::/64;2001:4860:4801:6f::/64;2001:4860:4801:70::/64;2001:4860:4801:71::/64;2001:4860:4801:72::/64;2001:4860:4801:73::/64;2001:4860:4801:74::/64;2001:4860:4801:75::/64;2001:4860:4801:76::/64;2001:4860:4801:77::/64;2001:4860:4801:78::/64;2001:4860:4801:79::/64;2001:4860:4801:80::/64;2001:4860:4801:81::/64;2001:4860:4801:82::/64;2001:4860:4801:83::/64;2001:4860:4801:84::/64;2001:4860:4801:85::/64;2001:4860:4801:86::/64;2001:4860:4801:87::/64;2001:4860:4801:88::/64;2001:4860:4801:90::/64;2001:4860:4801:91::/64;2001:4860:4801:92::/64;2001:4860:4801:93::/64;2001:4860:4801:c::/64;2001:4860:4801:f::/64;192.178.5.0/27;192.178.6.0/27;34.100.182.96/28;34.101.50.144/28;34.118.254.0/28;34.118.66.0/28;34.126.178.96/28;34.146.150.144/28;34.147.110.144/28;34.151.74.144/28;34.152.50.64/28;34.154.114.144/28;34.155.98.32/28;34.165.18.176/28;34.175.160.64/28;34.176.130.16/28;34.22.85.0/27;34.64.82.64/28;34.65.242.112/28;34.80.50.80/28;34.88.194.0/28;34.89.10.80/28;34.89.198.80/28;34.96.162.48/28;35.247.243.240/28;66.249.64.0/27;66.249.64.128/27;66.249.64.160/27;66.249.64.224/27;66.249.64.32/27;66.249.64.64/27;66.249.64.96/27;66.249.65.0/27;66.249.65.160/27;66.249.65.192/27;66.249.65.224/27;66.249.65.32/27;66.249.65.64/27;66.249.65.96/27;66.249.66.0/27;66.249.66.160/27;66.249.66.192/27;66.249.66.32/27;66.249.66.64/27;66.249.66.96/27;66.249.68.0/27;66.249.68.32/27;66.249.68.64/27;66.249.69.0/27;66.249.69.128/27;66.249.69.160/27;66.249.69.192/27;66.249.69.224/27;66.249.69.32/27;66.249.69.64/27;66.249.69.96/27;66.249.70.0/27;66.249.70.128/27;66.249.70.160/27;66.249.70.192/27;66.249.70.224/27;66.249.70.32/27;66.249.70.64/27;66.249.70.96/27;66.249.71.0/27;66.249.71.128/27;66.249.71.160/27;66.249.71.192/27;66.249.71.224/27;66.249.71.32/27;66.249.71.64/27;66.249.71.96/27;66.249.72.0/27;66.249.72.128/27;66.249.72.160/27;66.249.72.192/27;66.249.72.224/27;66.249.72.32/27;66.249.72.64/27;66.249.72.96/27;66.249.73.0/27;66.249.73.128/27;66.249.73.160/27;66.249.73.192/27;66.249.73.224/27;66.249.73.32/27;66.249.73.64/27;66.249.73.96/27;66.249.74.0/27;66.249.74.128/27;66.249.74.32/27;66.249.74.64/27;66.249.74.96/27;66.249.75.0/27;66.249.75.128/27;66.249.75.160/27;66.249.75.192/27;66.249.75.224/27;66.249.75.32/27;66.249.75.64/27;66.249.75.96/27;66.249.76.0/27;66.249.76.128/27;66.249.76.160/27;66.249.76.192/27;66.249.76.224/27;66.249.76.32/27;66.249.76.64/27;66.249.76.96/27;66.249.77.0/27;66.249.77.128/27;66.249.77.160/27;66.249.77.192/27;66.249.77.224/27;66.249.77.32/27;66.249.77.64/27;66.249.77.96/27;66.249.78.0/27;66.249.78.32/27;66.249.79.0/27;66.249.79.128/27;66.249.79.160/27;66.249.79.192/27;66.249.79.224/27;66.249.79.32/27;66.249.79.64/27;66.249.79.96/27" -description "Googlebot"

 

[Sebahat.hadzhi]

The Plesk CLI documentation page doesn't make any reference to the usage of semicolons versus spaces so what does that mean in the context of mixing IPs with CIDR ranges?

When you execute plesk bin ip_ban --help you can see:

--add-trusted "<ip>[;<ip>;...]" Adds IP addresses to the list of
trusted addresses. If `-description`
is provided, same description will be
applied to all IPs in batch.

In the example next to --add trusted you can see the semicolon. Each value separated with semicolon is treated as single entity be is an IP or a range.

The syntax of the command looks good to me. I executed it on a test environment and from a first glance it looks like everything is properly added to the whitelist. (Please note that I haven't went through all IP addresses/ranges one by one to confirm if they are added).