Resolved How to check details about POP3/IMAP traffic for single email

[MicheleB]

Recently I've upgraded my server from Plesk 11.5 to the last version 12.5.
Now I've a high POP3/IMAP traffic for a single domain (from 30 to 300 GB monthly).
I'd like to know if is possibile to find the single email (the domain has more 30+ emails) that use improperly the IMAP/POP3 service.

Plesk offer a tool for this?
Alternatively, is there any script/plugin?

Thanks.

 

[serverpoint_tom]

Hello,

Diagnose that mailbox size is what is causing your problem.

Unfortunately, Plesk doesn't have an option to check individual mailbox size. The simplest method to do this is as follows:

• Log into your server as a root or sudo user via SSH, then run the following command (be sure to replaceexample.com with your domain name, and username with the email user):

du -sh /var/qmail/mailnames/example.com/username/Maildir/

Thank you,

 

[MicheleB]

Ok, thanks but Plesk offer already the possibility to check the size about every single email (see "usage" column, in the "Mail" section).
I need to verify only the traffic POP3/IMAP for the single email and I don't think that the email size is the correct way to understand (many emails could occupy much space but be unused).

 

[MicheleB]

Do you know if exist a ssh command to filter/summarize only traffic POP3/IMAP for a single email in the mailog.log file?

I'd like to try to check the size for every single email in a single day, but manually it's impossible:
Dec 25 07:00:46 abcdomain postfix/qmgr[8890]: 7FGHF76K59: from=<[email protected]>, size=2891, nrcpt=1 (queue active)

 

[MicheleB]

or... is available an extension (I saw a lot of extensions in the catalog) to enable in Plesk more details about POP3/IMAP traffic?
Thanks

 

[trialotto]

@MicheleB,

It is not about size at all, it is about frequency AND incoming and/or outgoing traffic.

Traffic is often not generated by one or two mailaddresses: traffic spikes are often caused by huge numbers of small mails (a.k.a. spam).

Huge numbers of incoming traffic would lead to a huge mailbox.

I am pretty sure that you investigated that already by having a look at mailbox sizes in the Plesk panel. You probably did not find anything amazing.

Let´s assume that mailboxes are not huge (and if that is an incorrect assumption, just let me know).

Huge numbers of outgoing traffic would be visible in the traffic statistics, but that would not give you any clue about the culprit, i.e. the mailaddresses that are causing the traffic spikes.

A simple solution is to check your maillog visually via ssh.

After all, one or two mailaddresses sending out a lot of mails would be equivalent to having the same mailaddresse(s) on almost every block of 5 to 10 lines, since the increase in traffic (from 30 GB to 300 GB) would be equivalent to those mailaddresses sending out (small) messages almost every minute.

I am pretty sure that you will find the culprit(s) with a quick visual inspection of the maillog.

Let me know what you find (before I start dealing with the more undesirable scenario´s).

Regards!

 

[MicheleB]

I'm searching on maillog file and I noticed a new voice "saved mail to INBOX" and a new service "dovecot" that weren't present before to upgrade to Plesk 12.5 (from Plesk 11.5... when the POP3/IMAP traffic was good):
Dec 18 07:01:59 abcdomain dovecot: service=lda, [email protected], ip=[]. msgid=<2015121820150234567896.kJFDSWER@gt89>: saved mail to INBOX

Do you know if "dovecot" service and "saved mail to INBOX" could increase the POP3/IMAP traffic?
Thanks

 

[trialotto]

@MicheleB,

Dovecot is an IMAP server and Dovecot itself should not be causing the drastic increase in mail related traffic.

However, it can be the case that one of your customers uses a particular (i.e. stupid) IMAP setup in which the whole mailbox is downloaded entirely, each time the mail client connects.

This particular setup is rather exceptional, but it can occur if some programmatical mail client has been constructed (instead of using known libraries or mail clients).

In general, you should exclude this possibility from your list of potential culprits.

In short, Dovecot is normally not causing traffic spikes.

Can you be more specific about the composition of the mail related traffic? Is it mostly incoming or outgoing? And so on.

Regards....

 

[MicheleB]

Ok, thanks.
The problem is only with POP3/IMAP (incoming) traffic, while smtp/http/ftp is normal.
I've asked my hosting/server provider and told me that probably is a problem with a single email configuration (backup, synchronization, etc... how you said me in the previous post) but without give me more info how to try this email (in that domain, there are 50+ email accounts).

 

[trialotto]

@MicheleB,

If you know the specific domain, but not the specific mailbox, you can try to do the following.

Open a ssh console and run from the command line:

a) in order to identify the (often harmless) imap logins: grep -in "tfw dovecot: imap-login: Login: user=" /var/log/maillog

Note: the number is quite handy to determine how frequently a user is being logged in.

Note: a huge number of logins is not surprising and it is only a good sign, since it indicates a shutdown of the connection, reducing traffic.

Note: a huge number of logins for a particular mailaddress is not indicating something particular.

b) in order to identify the (often harmful) pop3 logins: grep -in "tfw dovecot: pop3-login: Login: user=" /var/log/maillog

Note: you will probably see that the list of mailaddresses is somewhat smaller than in the result from the command under point a.

c) have a look at the size of the mailboxes, resulting from the command in point b.

Note: you probably have found the culprit for the traffic spike by now and if you did not, have a look at mailbox sizes of the three most frequently occurring mailaddresses in point a.


Hope the above helps a little bit.

Please note that there are more easy ways to do this (i.e. scripting) and/or that it takes more analysis to investigate the issue (for instance, a not closed connection can cause traffic).

For the time being, the above is beyond the scope of the topic, we will return to that, if and only if necessary.

Can you report back?

Regards....

 

[MicheleB]

Hi,
I'm not really practical with SSH command line.

If I use these instructions not give me any results:

1) grep -in "tfw dovecot: imap-login: Login: user="/var/log/maillog

2) grep -in "tfw dovecot: pop3-login: Login: user="/var/log/maillog

 

[trialotto]

MicheleB,

Put a space between " and /var/log/maillog, when running the command.

 

[MicheleB]

I tried but for either commands receive this result:
[Exit 1]

root@abcdomain:~# grep -in "tfw dovecot: imap-login: Login: user=" /var/log/maillog
[Exit 1]

root@abcdomain:~# grep -in "tfw dovecot: pop3-login: Login: user=" /var/log/maillog
[Exit 1]

 

[trialotto]

MicheleB,

Can you check where your maillog is located? It can be in /var/log/ or in /usr/local/psa/var/log/.

Run the command: find / -name maillog (and the location can be used to run the grep command).

 

[MicheleB]

"find / -name maillog" give me these results:
/opt/psa/var/log/maillog
/var/log/maillog


... I tried with "/opt/psa/var/log/maillog" and receive always "Exit 1":
root@abcdomain:~# grep -in "tfw dovecot: imap-login: Login: user=" /opt/psa/var/log/maillog
[Exit 1]

root@abcdomain:~# grep -in "tfw dovecot: pop3-login: Login: user=" /opt/psa/var/log/maillog
[Exit 1]

 

[trialotto]

Well, then check your maillog visually, by just opening the log file.

 

[MicheleB]

I've found the problem... pingdom monitoring service.
Too connections pop3/imap/smtp from pingdom.
I've interrupt the monitoring service (each with 1 minute interval) and the traffic now is normal (from 20 to 1 GB daily).

I also activated Fail2Ban to avoid brute force attack and now the traffic is return completely normal.

 

[trialotto]

@MicheleB,

Ehm, pingdom monitoring service?? That should not affect mailboxes or mail related traffic.

Anyway, since your problem is resolved, I am quite happy.

Regards...

 

[ahmet hassan]

When I read this correctly, Plesk shows you the outgoing IMAP traffic. This normally means that an IMAP client connects to your server and checks/receives mails.

This could also be caused by backup or synchronization scripts (I guess you would be aware of those).

My own Plesk installation uses courier-imap with logging disabled at default. I recommend to enable it according to this Plesk Knowledge Base article: Enabling POP3/IMAP logging in CourierIMAP.

You then should be able to have a look at the logs and find out what is happening there.

Another approach could be running tcpdump or tshark on the IMAP port and look what is happening. The disadvantage of this method is that you would need proper timing.

 

[MicheleB]

I reopen this post only to clarify the position of pingdom alert service.
I've found the real cause of the problem... a single email used with a script for synchronization activity.
Pingdom was absolutely "ok"... I'm sorry to have created bad publicity to this great monitoring service.

I hope to see soon on Plesk more controls about single domain activity (not only single email stats but also CPU usage for a specific domain and other things useful to found any problems).