Whitelisted IPs are stored in /etc/fail2ban/jail.local
Would make sense just to download that file, uninstall fail2ban, clean up any errant files, install fail2ban, stop fail2ban, upload the file (overwriting preserves permissions) and finally start fail2ban.
I did find (at a later date) that...