The 'memory allocation' issue is usually related to the number of rules set into the firewall. Plesk does not control the network traffic, the firewall extension or fail2ban just configures the iptables firewall rules provided by the operating system vendor, which works within the kernel. I suggest you consider contacting the OS vendor in order to see if this limit can be circumvented somehow.