Dovecot: POP3 works in mail client, IMAP doesn't

[Florian]

Hi everyone,

I got a server up and running the other day with Plesk preinstalled. So I got myself a license since I really like the control panel thus far. Everything works with one exception: emails with Dovecot.

• Sending and receiving emails does work with the Roundcube webmailer
• Receiving emails via POP3 does work in Thunderbird
• Receiving emails via IMAP (or even just setting up the account) in Thunderbird doesn't work

According to Plesk, IMAP is running for that particular mail account and I can successfully connect to Dovecot via telnet on port 143.
I gave Thunderbird the following settings (anonymous and actually photoshopped since my Thunderbird's all German). The values for Port, SSL and Authentification were chosen automatically but do match the ones Plesk provides.

So according to Thunderbird my login credentials are wrong. They actually aren't, I double checked and I can log into the webmailer with them.

According to the Dovecot log files I didn't even try to authenticate.

Mar  1 18:22:34 #server_name# postfix/smtpd[15206]: connect from p54B0F509.dip0.t-ipconnect.de[#my_ip#]
Mar  1 18:22:34 #server_name# postfix/smtpd[15210]: connect from p54B0F509.dip0.t-ipconnect.de[#my_ip#]
Mar  1 18:22:34 #server_name# postfix/smtpd[15206]: improper command pipelining after EHLO from p54B0F509.dip0.t-ipconnect.de[#my_ip#]: QUIT\r\n
Mar  1 18:22:34 #server_name# postfix/smtpd[15206]: disconnect from p54B0F509.dip0.t-ipconnect.de[#my_ip#]
Mar  1 18:22:34 #server_name# postfix/smtpd[15210]: improper command pipelining after EHLO from p54B0F509.dip0.t-ipconnect.de[#my_ip#]: QUIT\r\n
Mar  1 18:22:34 #server_name# postfix/smtpd[15210]: disconnect from p54B0F509.dip0.t-ipconnect.de[#my_ip#]
Mar  1 18:22:34 #server_name# dovecot: imap-login: Aborted login (no auth attempts in 0 secs): user=<>, rip=#my_ip#, lip=#server_ip#, session=<#crpytic_sessionname#>
Mar  1 18:22:34 #server_name# dovecot: imap-login: Aborted login (no auth attempts in 0 secs): user=<>, rip=#my_ip#, lip=#server_ip#, session=<#crpytic_sessionname#>

It seems Thunderbird's trying to log me in completely without my user name. Or does it have to do with the entries above the actual login?

Additionally I tried (in opposition to the Plesk instructions) to use the full mail address at incoming and outgoing username.
I also tried all the same procedures with other domains and mail accounts. The result is the same as described above everywhere.

And still what confuses me the most is that everything's running just fine using POP3 instead of IMAP.

Sadly Google couldn't help me so far. Does anyone have an idea how to approach this problem?

 

[trialotto]

@Florian

Your password or username seems to be invalid: check the password first.

 

[Florian]

The used credentials should be correct since they're the same ones that I use to successfully log into the webmailer.

 

[Lloyd_mcse]

@Florian

The usernames should be [email protected], and ...

IMAP Port 143 is StartTLS+Normal Password
Or
IMAP Port 993 is SSL/TLS+Encrypted Password

Also you should use...

Submission Port 587 StartTLS+Encrypted Password

Rather than port 25, but that's another topic
I hope that helps get you connecting.
Regards

Lloyd

 

[Florian]

Unfortunately that didn't help.
IMAP port 993: After testing the connection Thunderbird states a "Thunderbird failed to find the settings for your email account."
IMAP port 143: After testing the connection Thunderbird overrides "Normal password" with "Encrypted password" in IMAP and "Encrypted password" with "Normal password" in SMTP and then fails to connect due to the credential problems stated above.

I adjusted the logs to be a little more verbose this time but it seems to me that the problem is basically the same as before.

First attempt (port 993):

Mar  3 10:37:07 #server_name# dovecot: imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges
Mar  3 10:37:07 #server_name# dovecot: imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges
Mar  3 10:37:07 #server_name# dovecot: auth: Debug: auth client connected (pid=31615)
Mar  3 10:37:07 #server_name# postfix/smtpd[31614]: connect from provider_domain.tld[#my_ip#]
Mar  3 10:37:07 #server_name# dovecot: imap-login: Aborted login (no auth attempts in 0 secs): user=<>, rip=#my_ip#, lip=#server_ip#, session=<#cryptic_sessionname#>
Mar  3 10:37:07 #server_name# postfix/smtpd[31614]: improper command pipelining after EHLO from provider_domain.tld[#my_ip#]: QUIT\r\n
Mar  3 10:37:07 #server_name# postfix/smtpd[31614]: disconnect from provider_domain.tld[#my_ip#]

Second attempt (port 143):

Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges
Mar  3 10:34:10 #server_name# postfix/smtpd[31603]: connect from provider_domain.tld[#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x10, ret=1: before/accept initialization [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: before/accept initialization [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth
Mar  3 10:34:10 #server_name# dovecot: auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x2002, ret=-1: unknown state [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x2002, ret=-1: unknown state [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libauthdb_plesk.so
Mar  3 10:34:10 #server_name# dovecot: auth: Debug: Read auth token secret from /var/run/dovecot/auth-token-secret.dat
Mar  3 10:34:10 #server_name# dovecot: auth: Debug: auth client connected (pid=31602)
Mar  3 10:34:10 #server_name# postfix/smtpd[31603]: disconnect from provider_domain.tld[#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x20, ret=1: SSL negotiation finished successfully [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL: where=0x2002, ret=1: SSL negotiation finished successfully [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Warning: SSL alert: where=0x4004, ret=560: fatal unknown CA [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Debug: SSL alert: close notify [#my_ip#]
Mar  3 10:34:10 #server_name# dovecot: imap-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=#my_ip#, lip=#server_ip#, TLS: SSL_read() failed: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca: SSL alert number 48, session=<#cryptic_sessionname#>

EDIT: Just saw the "fatal unknown CA". Could that cause the problem?
And why does it occur - I use the default self-signed certificates that ship with Plesk but they should actually work, shouldn't they?

 

[trialotto]

@Florian,

It seems to me (but I can be mistaken) that something is wrong with postfix, given the "improper command pipelining".

I would strongly recommend to switch to Qmail and switch back to Postfix afterwards, in order to exclude any error related to Postfix.

You can use the installer or the command line autoinstaller (the latter works faster and is more verbose), note that the switch between Qmail/Postfix does not affect your mail data.

Just let us know whether the problem persists after the switch (and provide some output, that could be insightful).

Regards.....

PS You should also be aware that you can only test with the settings provided by @Lloyd_mcse, with port 143 (incoming, IMAP) and port 587 (outgoing, SMTP).

 

[Florian]

First: Thanks for all the answers and help attempts so far!

Second: Sadly switching to Qmail didn't fix the problem. The mail log pretty much states the same:

Mar  3 21:44:37 #server_name# dovecot: imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges
Mar  3 21:44:37 #server_name# dovecot: imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges
Mar  3 21:44:37 #server_name# dovecot: auth: Debug: auth client connected (pid=18348)
Mar  3 21:44:37 #server_name# dovecot: imap-login: Aborted login (no auth attempts in 0 secs): user=<>, rip=#my_ip#, lip=#server_ip#, session=<#cryptic_sessionname#>

 

[J-F Brouillette]

I would strongly recommend to switch to Qmail and switch back to Postfix afterwards, in order to exclude any error related to Postfix.

@Florian : Just to be sure, you said that you switched to Qmail but did you switch back to Postfix?

 

[Florian]

Oh sorry, I didn't realize I actually just should switch to kind of reset Postfix.
I did now, though it didn't do me any favor.

Mar  3 22:28:00 #server_name# dovecot: imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges
Mar  3 22:28:00 #server_name# dovecot: imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges
Mar  3 22:28:00 #server_name# dovecot: auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth
Mar  3 22:28:00 #server_name# dovecot: auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth
Mar  3 22:28:00 #server_name# dovecot: auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libauthdb_plesk.so
Mar  3 22:28:00 #server_name# dovecot: auth: Debug: Read auth token secret from /var/run/dovecot/auth-token-secret.dat
Mar  3 22:28:00 #server_name# postfix/smtpd[28828]: connect from provider.tld[#my_ip#]
Mar  3 22:28:00 #server_name# dovecot: auth: Debug: auth client connected (pid=28827)
Mar  3 22:28:00 #server_name# postfix/smtpd[28828]: disconnect from provider.tld[#my_ip#]
Mar  3 22:28:00 #server_name# dovecot: imap-login: Aborted login (no auth attempts in 0 secs): user=<>, rip=#my_ip#, lip=#server_ip#, session=<#cryptic_sessionname#>

 

[trialotto]

@Florian,

Sorry for the delay, but can you give the output of the command: dovecot -n

Regards....

 

[Florian]

Of course. Here you go:

# 2.2.18: /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.8 (0c4ae064f307+)
# OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.3 ext4
auth_debug = yes
auth_debug_passwords = yes
auth_mechanisms = plain login digest-md5 cram-md5 apop
auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890&.-_@'
auth_verbose = yes
auth_verbose_passwords = plain
disable_plaintext_auth = no
first_valid_uid = 30
imap_client_workarounds = delay-newmail
imap_logout_format = rcvd=%i, sent=%o
mail_debug = yes
mail_home = /var/qmail/mailnames/%Ld/%Ln
mail_location = maildir:/var/qmail/mailnames/%Ld/%Ln/Maildir
mail_log_prefix = "service=%s, user=%u, ip=[%r]. "
mail_plugins = " quota"
managesieve_logout_format = rcvd=%i, sent=%o
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate imapflags notify
namespace inbox {
  inbox = yes
  location =
  prefix = INBOX.
  separator = .
}
passdb {
  driver = plesk
}
plugin {
  quota = maildir:User quota
  quota_grace = 0
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
  sieve_extensions = +notify +imapflags
}
pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
pop3_logout_format = rcvd=%i, sent=%o, top=%t/%p, retr=%r/%b, del=%d/%m, size=%s
protocols = imap pop3 sieve
service auth-worker {
  group =
  user =
}
service auth {
  group =
  unix_listener auth-userdb {
    group = popuser
    mode = 0600
    user = popuser
  }
  user =
}
service imap {
  process_limit = 1024
  service_count = 1
}
service pop3 {
  process_limit = 1024
  service_count = 1
}
ssl_cert = </etc/dovecot/private/ssl-cert-and-key.pem
ssl_cipher_list = HIGH:!aNULL:!MD5
ssl_key = </etc/dovecot/private/ssl-cert-and-key.pem
ssl_protocols = TLSv1 TLSv1.1 TLSv1.2
userdb {
  args = uid=popuser gid=popuser
  driver = static
}
verbose_ssl = yes
protocol imap {
  mail_plugins = " quota imap_quota"
}
protocol pop3 {
  pop3_uidl_format = UID%u-%v
}
protocol lda {
  mail_plugins = " quota sieve"
}

 

[trialotto]

@Florian

I think that you also have to have a look at /var/log/auth.log and, if required, post some relevant output.

However, I have the impression that you currently should not have any issue with postfix and/or dovecot (i.e. dovecot seems to be fine), so we cannot exclude that a firewall rule, being set by yourself or Fail2Ban, is currently blocking any login attempt from your personal IP.

Please check the firewall and Fail2Ban and DO whitelist the personal IP, before we can proceed.

Let me know what happens.....

Regards

 

[trialotto]

@Florian,

Also recheck the settings of the mail client, i.e. Thunderbird, since the logs do not exactly indicate that SSL or TLS has been used or has been succesful.

Regards....

 

[Florian]

• auth.log doesn't state anything except the SSH login when I downloaded the auth.log via SCP
• I don't have Fail2Ban running on the server at the moment.
• whitelisting the server's IP on my local machine didn't help
• Thunderbird settings seem to be fine. I don't think TB's settings are the problem since at work I tried things with a fresh copy of Thunderbird (and another OS and another IP btw.) and got the exact same results.

Question to make my life a bit easier: Should I direct the MX DNS record to mydomain.tld or mail.mydomain.tld (i.e. does it even matter?).
It was the mail subdomain by default before I had the server installed freshly with another control panel software.
Anyway it shouldn't matter too much since at the moment I'm testing two Mail accounts of two TLDs on the same server in parallel, one having the MX record for the TLD, one having it for the subdomain.
But I could give up on all the double testing if I knew exactly if there was a right or wrong way to do that.

 

[trialotto]

@Florian,

Note that I intended to get your local machine´s IP not blocked by any firewall rule on the server side (seems to be the case that you did it the other way around).

Also note that it should not matter whether mail.domain.tld or mydomain.tld is being used, as long as both are referring to the same IP AND the MX records are properly configured.

By the way, what I do not get is the following: in the first post, you seem to have post a screenshot from Outlook and we are now discussing Thunderbird.......eh?

In essence, what happens is that the mail client does not even attempt to connect, as is indicated by:

imap-login: Aborted login (no auth attempts in 0 secs)

In short, it is definitely the mail client or something else prohibiting that the mail client is connecting.

The usual suspects are

- a firewall setting (can also be induced by Fail2Ban): checked, not the case, so it seems
- a dovecot issue: no reason to assume that this is the case, certainly when taking into account that dovecot has a standard configuration and that pop3-login seems to be possible,
- a TLS/SSL issue: not the case,
- a certificate issue: certainly not the case, if pop3-login works,
- a bug in the client (i.e. Thunderbird): this should not be the case, if the latest update is installed,
- a misconfiguration of the client: not properly checked (yet),

and, given the above (and the absence of Fail2Ban), it is very likely something with the mail client, in specific the settings.

You can try to connect with Outlook, using IMAP (if that works, then it is definitely a misconfiguration in Thunderbird, if the latest Thunderbird update has been installed).

Regards......

 

[trialotto]

@Florian

I forgot to mention that a German language setting also belongs to the set of usual suspects: umlauts are not the favourite things of any system.

You can work-around a potential language issue by installing an English version of Thunderbird and/or changing the password to a "normal" password.

Regards.....

 

[Florian]

@trialotto

• It's always been Thunderbird, from the first screenshot until now.
• I don't use umlauts in any of the relevant settings or names. Also the Thunderbird installation at work was actually in English to exclude all eventualities.
• I'm not sure about normal/encrypted passwords since (as stated waay above) Thunderbird swaps them around if I click "Re-test" giving IMAP 143 an encrypted password and SMTP 587 a normal one.
• This could be me having a strange perception but I have the feeling that if I click on "Done" in the account settings Thunderbird immediately complains about the credentials. It even feels too short to have tried to authenticate and await a proper response. But I guess that's pretty obvious as you've stated above - TB doesn't even try to log in.
• I can't get Windows Mail to work with the account either.
  Nevertheless since the moment I tried to log in with Windows mail the dovecot log is telling me users named "william", "adm", "sync", "operator" or "shutdown" (probably even more) are trying to log in from an IP in Sweden. Which is kind of frightening me. Enabled Fail2ban, just in case.

 

[Florian]

The situation kind of scares me a little bit which is why I just disabled the dovecot service for the moment.

 

[trialotto]

@Florian,

Do not disable dovecot, that makes you even more vulnerable.

Yep, one of the unusual suspects is an overload of logins, which causes the maximum number of connections to be used, hence not allowing other users to login.

This is very rare, but happens with brute forcing and/or distributed attacks. Strange to see this happen.

You are right to be frightened, you are very likely to be under attack, given the names of the users (which indicates some kind of dictionary type attack).

Please make sure that

a) you manually block all bad IPs in the firewall (use the Plesk Firewall, go to Tools & Settings > Security), since Fail2Ban does take some time to ban an mail related bad IP,

b) make Fail2Ban more tight: set the maxretry value to 2 (or even 1) on the jails

- plesk-dovecot
- the one concerning postfix
- recidive

and make sure that the jails are activated,

c) you disable the debug mode,

d) decrease the maximum number of connections, go to "Tools & Settings > Mail Server settings > Mail" and

- reduce the value of "Maximum number of connections (IMAP, POP3, IMAP over SSL, or POP3 over SSL), take something like 512
- reduce the value of "Maximum number of connections for a user per IP address" to 5 or 10

and if the unknown users are disappearing from the maillog, just revert the values,

e) you inspect the mail queue, in case somebody already entered the system,

f) you inspect the maillog for succesful entries of bad IPs or unknown users (and track whether IMAP or POP3 or both have been used, in the case of bad logins).


Note that it often helps to restart the mail service a couple of times AND it is even better to stop the mail service during a couple minutes: in essence, some hack scripts will give up.

The most important part is that you add all bad IPs in the firewall, that is good for now and safeguards you to some extent in the near and far future.

Finally, have a good look at your system, since it can be compromised already: I stated earlier that this situation is rather strange and it can be the case that a mail attack is a mask for some more elaborate hack in the background.

Hope the above helps.

Regards.........

 

[Florian]

Okay.
a) So I blocked the IP (was only one all the time) in the Firewall. According to a WhoIs it belongs to a VPS provider. I guess I should contact them to tell them of the unusual actions?
b) I adjusted Fail2Ban and
c) disabled debug mode.
d) I made the suggested changes. Didn't revert them by now.
e) Mail queue is empty.
f) According to the log there was only one bad IP which couldn't get in.

I already disabled mail service earlier the night (before your post), attacks were not going on after re-enabling it.

So I couldn't find any indications so far that the system's compromised. But since the server is rather freshly set up it wouldn't be too much of an upset to just do a complete reinstall.
(How much) would you recommend to do that?