Question Autodiscover

[Janko1000]

Hi,

is there any Documentation available for the Autodiscover Function?
How does it work? Works it for Thunderbird, Outlook and Apple Mail?



Thank you and have a nice weekend

 

[Werner99]

i tested it with gmail, thunderbird and outlook 2016

gmail and thunderbird works perfect for me, on outlook 2016 it gives me an little issue but nothing serious.

"The Name on the Certificate is not valid or is not equal to the Website Name" but it saying yes to continue everything is fine after.
Sorry for bad translation i translated it from german

 

[grimley]

Works with Outlook here. A certificate mismatch notification will appear. I resolved this by using by panel hostname for incoming and outgoing server settings in Outlook (performed via Control Panel / Mail after initial setup). Outlook on Android and on iPad OS don't display certificate errors.

 

[Ales]

To what did the autodiscover set the incoming and outgoing server initially?

Which mail and IMAP server are you using?

 

[seqoi]

I resolved this by using by panel hostname for incoming and outgoing server settings in Outlook (performed via Control Panel / Mail after initial setup).

But that mean you are using your server certificate (top certificate so to speak) - not user domain certificate. That's why you don't have error. This is working same as in Onyx. It's not wrong way of doing it but i would like to use new SNI feature and i want my users can use their domain names for incoming and outgoing servers.

Ales is there any way we can avoid this "The Name on the Certificate is not valid or is not equal to the Website Name" error? I mean to not appear at all?

 

[Werner99]

it sets it to domain.com so no imap.domain.com or smtp.domain.com, and with that no certificate issue aside the one in outlook 2016

To what did the autodiscover set the incoming and outgoing server initially?

Which mail and IMAP server are you using?

 

[Ales]

If the SNI is working with the "example.com" in the certificate and if the "example.com" is set in Outlook 2016's incoming and outgoing server settings, I wouldn't expect an error... This should be investigated further.

I admit I haven't researched the SNI inner workings in the released Obsidian yet. I was hoping that as Obsidian progressed towards release candidates and the actual release, the mail SNI would get more features... But apparently the SNI is only set up to work with Postfix and Dovecot on linux and the domain in the SNI is automatically set to "example.com". So no choice of "mail.example.com" and/or "smtp.example.com" and/or "imap.example.com" and/or "pop3.example.com", ...

This doesn't work for us, so basically we're about in the same situation as we were with Onyx.

 

[seqoi]

But apparently the SNI is only set up to work with Postfix and Dovecot on linux and the domain in the SNI is automatically set to "example.com". So no choice of "mail.example.com" and/or "smtp.example.com" and/or "imap.example.com" and/or "pop3.example.com", ...

This doesn't work for us, so basically we're about in the same situation as we were with Onyx.

If "example.com" is working without any error i couldn't care less for "smtp.example.com". IMHO example.com is even more straightforward from the client perspective. He doesn't have to remember all these imap,pop etc things. So if that works without any error then that's fine.

But my stand is that Obsidian shouldn't return any error in autodiscovery. "The Name on the Certificate is not valid or is not equal to the Website Name" sounds very confusing for customer. I would really hate spending my time replying or answering my phone calls and explaining my customers "hey it's ok, your data is safe just hit ok". This should simply work straight out of the box. cPanel solved this years ago. Not trying to start a discussion between cPanel vs Plesk vs XY - it's just i am stumped noone is taking care for such rudimentary workflow things.

On top of that we waited for SNI like literally for years. Plesk even advertise it as a highlight feature. Yet it seems like it's somehow overseen, untested and not working entirely.

I guess Plesk team have a lot of tasks at their table.

 

[Ales]

Just to explain further, we have a considerable amount of customers that have most of the services controlled by Plesk, but their actual web hosting is on a different platform. That's why simply using "example.com" as a mail server name can't work for us.

Well, that, and the fact that we don't use Postfix...

 

[mr-wolf]

I have my own autodiscovery working for years.
I hope Plesk's system will not interfere with what I had.
Now I need to investigate how they are achieving it. Not some task I was waiting for.

Mine doesn't rely on SNI which is not supported by many mail-clients anyway.

 

[sebgonzes]

Well, I have testit in our case, work with outlook but without any option of SSL!
Don't work with thunderbird that detect "mail.domain.com" and not valid SSL (if had exception work, but it's not the goal!)
Work on andriod mail....
Is there somewhere to review this in configuration?

 

[Janko1000]

I have my own autodiscovery working for years.
Mine doesn't rely on SNI which is not supported by many mail-clients anyway.

Please tell more

 

[mr-wolf]

Please tell more

It boils down to having a special cname for each domain and an SRV-record.

You need to have a wildcard certificate registered to your domain (assuming you're some kind of hosting provider).

Let's say I have the domain "wolf.com".
Then I have a wildcard certificate "*.wolf.com"

Each client of mine has a "mail.<domain>" record which obviously points to one of your Plesk servers.
I will make sure that each domain also has a cname pointing to that record.

Let's say you're a client of mine with the domain janko.com
Your mail-server will be mail.janko.com
If you connect to the mail-server you will only get 1 certificate, namely *.wolf.com

That will not match with "mail.janko.com"
That's why we need a cname.
I automatically create a cname in my own zone for each of my client's domains.

For you that would be the cname

janko-com.wolf.com. IN CNAME mail.janko.com.

So that's the domain you need to use to connect with one of my mailservers.
Now we need an autodiscovery for that...

_autodiscover._tcp.janko.com. IN SRV 0 1 443 janko-com.wolf.com.

Which means that Outlook will connect to https://janko-com.wolf.com/Autodiscover/Autodiscover.xml

Nginx will detect the regular expression in the domain with this directive

server_name ~^[a-z0-9-]+[a-z0-9]-[a-z0-9]+\.wolf.com$;

Because autoconfig (Thunderbird) does not need https you do not need to have a certificate and it becomes simpler.

autoconfig.janko.com. IN CNAME mail.janko.com.

cat /etc/nginx/conf.d/zz095_autodiscover.conf

server {
    listen 8.8.8.8:80;
    server_name ~^autoconfig\.[a-z0-9-]+\.[a-z0-9-]+$;

    root  /var/www/autoconfig_autodiscover;

    index index.html;
    error_log /var/log/nginx/autoconfig_autodiscover/error.log;
    access_log /var/log/nginx/autoconfig_autodiscover/access.log combined;

    location ~* /mail/config-v1.1.xml {
      try_files $uri /config-v1.1.xml.php?$args;
      rewrite ^(.+)$ /config-v1.1.xml.php?$1 last;
    }

   location ~ /config-v1\.1\.xml\.php$ {
     try_files $uri =404;
     include /etc/nginx/fastcgi_params;
     fastcgi_pass 127.0.0.1:9000;
     fastcgi_index index.php;
     fastcgi_param SERVER_FQDN  $host;
     fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
     fastcgi_param PATH_INFO $fastcgi_script_name;
     fastcgi_intercept_errors on;
   }
}


server {
    listen 8.8.8.8:443 ssl http2;
    # mydomain.com = ^mydomain-com.wolf.com$
    server_name ~^[a-z0-9-]+[a-z0-9]-[a-z0-9]+\.mr-wolf.nl$;
    root  /var/www/autoconfig_autodiscover;

    error_log /var/log/nginx/autoconfig_autodiscover/error.log;
    access_log /var/log/nginx/autoconfig_autodiscover/access.log combined;

    ssl_certificate             /root/.ssh/2018/2018.wildcard.wolf.com.pem;
    ssl_certificate_key         /root/.ssh/2018/2018.wildcard.wolf.com.key;

    ssl_stapling on;
    ssl_stapling_verify on;

    add_header Strict-Transport-Security $hsts_isd_header always;
    #add_header         Content-Security-Policy         "default-src 'self';" always;
    add_header          Referrer-Policy                 strict-origin-when-cross-origin always;
    add_header          X-Frame-Options                 SAMEORIGIN always;
    add_header          X-XSS-Protection                "1; mode=block" always;
    add_header          X-Content-Type-Options          nosniff always;

    ssl_dhparam         /etc/dhparam/dhparam4096.pem;
    ssl_session_timeout 5m;

    ssl_protocols       TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers         HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers   on;

    client_max_body_size 1m;

    index index.php index.html index.htm;

    location ~* /autodiscover/autodiscover.xml {
      try_files $uri /autodiscover.xml.php?$args;
      rewrite ^(.+)$ /autodiscover.xml.php?$1 last;
    }

   location ~ /autodiscover\.xml\.php$ {
     try_files $uri =404;
     include /etc/nginx/fastcgi_params;
     fastcgi_pass 127.0.0.1:9000;
     fastcgi_index index.php;
     fastcgi_param SERVER_FQDN  $host;
     fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
     fastcgi_param PATH_INFO $fastcgi_script_name;
     fastcgi_intercept_errors on;
    }
}

cat /var/www/autoconfig_autodiscover/autodiscover.xml.php

<?php
preg_match("/\<EMailAddress\>(.*?)\<\/EMailAddress\>/", file_get_contents("php://input"), $matches);
$server = $_SERVER["HTTP_HOST"];

//set Content-Type
header("Content-Type: application/xml");
?>
<?php echo '<?xml version="1.0" encoding="utf-8" ?>'; ?>
<Autodiscover xmlns="http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006">
<Response xmlns="http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a">
<Account>
<AccountType>email</AccountType>
<Action>settings</Action>
<Protocol>
<Type>IMAP</Type>
<Server><?php echo $server; ?></Server>
<Port>993</Port>
<DomainRequired>off</DomainRequired>
<LoginName><?php echo $matches[1]; ?></LoginName>
<SPA>off</SPA>
<SSL>on</SSL>
<AuthRequired>on</AuthRequired>
</Protocol>
<Protocol>
<Type>SMTP</Type>
<Server><?php echo $server; ?></Server>
<Port>587</Port>
<DomainRequired>off</DomainRequired>
<LoginName><?php echo $matches[1]; ?></LoginName>
<SPA>off</SPA>
<AuthRequired>on</AuthRequired>
<UsePOPAuth>off</UsePOPAuth>
<SMTPLast>off</SMTPLast>
<Encryption>TLS</Encryption>
<TLS>on</TLS>
</Protocol>
</Account>
</Response>
</Autodiscover>

cat /var/www/autoconfig_autodiscover/config-v1.1.xml.php

<?php
header ("Content-Type:text/xml");
$mail = $_GET['emailaddress'];
$host = $_SERVER["HTTP_HOST"];

$domain = str_replace("autoconfig.","",$host);
$server = str_replace(".","-",$domain) . ".wolf.com";

echo <<<EOP
<?xml version="1.0"?>
<clientConfig version="1.1">
<emailProvider id="{$domain}">
<domain>{$domain}</domain>
<displayName>{$mail}</displayName>
<displayShortName>{$mail}</displayShortName>
<incomingServer type="imap">
<hostname>{$server}</hostname>
<port>143</port>
<socketType>STARTTLS</socketType>
<username>{$mail}</username>
<authentication>password-cleartext</authentication>
</incomingServer>
<outgoingServer type="smtp">
<hostname>{$server}</hostname>
<port>587</port>
<socketType>STARTTLS</socketType>
<authentication>password-cleartext</authentication>
<username>{$mail}</username>
</outgoingServer>
</emailProvider>
</clientConfig>
EOP;

 

[Powie]

Hint: I think that Outlook is buggy. If the subdomain mail.<domain> was present in the DNS, Outlook ignored autodiscover and try's to use mail. . After we have deleted the mail. subdomain from DNS and also set the mx to the plesk hostname ( normally the better way) then autoconfiguration with outlook was working.

Maybeee somone can confirm this

 

[mr-wolf]

Never noticed anything like that, but I can confirm there's a lot wrong with Microsoft's autodiscovery process.
You would think that having an SRV-record should get precedence over any other record, but it's not.
We have a client that has its website with another provider, but even if there's no autodiscover DNS record other than the SRV-record, outlook will first check the webserver of their main domain.
That other provider is running cPanel and is leading an outlook mail client to itself.

It does this because the webserver responds to https://<domain>/autodiscover/autodiscover.xml


They also have a "secret test". The first outlook does is check Microsoft itself (Office 365). If the domain is registered there it will not follow the autodiscovery process.

 

[mr-wolf]

If you want to prove it does what you say, you can turn logging on at your DNS-server after setting TTL to 1 minute.

You would then prove the behaviour of your version of outlook. I noticed a lot of changes in the autodiscovery behaviour in the latest Outlook. Even minor versions

 

[Anthony]

Hello,

How does it work? Works it for Thunderbird, Outlook and Apple Mail?

The Microsoft Outlook and Thunderbird are officially supported only. The mail autodiscovery for Apple is always requests configuration from Apple servers. Right now we are working on how we can correctly support Apple Mail clients.

But that mean you are using your server certificate (top certificate so to speak) - not user domain certificate. That's why you don't have error. This is working same as in Onyx. It's not wrong way of doing it but i would like to use new SNI feature and i want my users can use their domain names for incoming and outgoing servers.

The Plesk Mail autodiscovery feature relies on <website.com> name and not to server hostname. That's why it's important to have properly set SSL/TLS certificate for domain (expected for each website nowadays) and use the SNI feature. Otherwise, you could face minor warnings when a mail client defines an SSL certificate (from hostname) which is not valid to be used for the domain configuration.

Hint: I think that Outlook is buggy. If the subdomain mail.<domain> was present in the DNS, Outlook ignored autodiscover and try's to use mail. . After we have deleted the mail. subdomain from DNS and also set the mx to the plesk hostname ( normally the better way) then autoconfiguration with outlook was working.

Plesk Mail autodiscovery not needed any special CNAME or SRV records in the domain's DNS zone as auto-discover using redirecting proxy on the websites and server webservers configuration files level. I've attached a PDF file with some details on how this feature has been implemented. Hopefully, it will shed light on the current mail auto-discover configuration.

 

[seqoi]

Hello,

I've attached a PDF file with some details on how this feature has been implemented. Hopefully, it will shed light on the current mail auto-discover configuration.

This is great thank you!

 

[mr-wolf]

@Anthony
I was not describing how Plesk has implemented autodiscovery. I have implemented autodiscovery 2 years ago myself and I do not plan changing that to Plesk's system.

The buggy autodiscovery process applies exactly the same on your as on my system. It's done conceptually wrong and is intended to work its best with Microsoft servers.
If your domain is registered at Office365 it will take precedence over everything. Even when someone only tried to register that domain there.

It's not that my system has no disadvantage, but I see 2 major disadvantages on the way Plesk has implemented it.
It relies on SNI (Only recent versions of Outlook use SNI) and it relies on LetsEncrypt certificates. Furthermore it also expects the webserver, mail server and DNS-server to be on the same server. This is not always the case and Plesk doesn't work well in some other areas either (dkim, letsencrypt wildcards).

I do however think that the way I implemented it is not suited for Plesk for their customers.

There are 4 ways of autodiscovery.
There are (afaik) only 3 mail clients that make use of autodiscovery and they all use a different one.
1 system isn't even used,

• RFC6186 (no mail client is using it) RFC 6186 - Use of SRV Records for Locating Email Submission/Access Services
  
• Microsoft Autodiscovery (only Outlook)
• Mozilla Thunderbird
• Apple Mail (I was unable to find info on it, I believe it involves a subscription @Apple)

How nice autodiscovery looks at first sight, it will become disappointing as most mail clients are not supported. They also don't support all the options. You can't, for instance, set the prefix of IMAP.

 

[sebgonzes]

Hello,

The Microsoft Outlook and Thunderbird are officially supported only. The mail autodiscovery for Apple is always requests configuration from Apple servers. Right now we are working on how we can correctly support Apple Mail clients.



The Plesk Mail autodiscovery feature relies on <website.com> name and not to server hostname. That's why it's important to have properly set SSL/TLS certificate for domain (expected for each website nowadays) and use the SNI feature. Otherwise, you could face minor warnings when a mail client defines an SSL certificate (from hostname) which is not valid to be used for the domain configuration.



Plesk Mail autodiscovery not needed any special CNAME or SRV records in the domain's DNS zone as auto-discover using redirecting proxy on the websites and server webservers configuration files level. I've attached a PDF file with some details on how this feature has been implemented. Hopefully, it will shed light on the current mail auto-discover configuration.

@Anthony is there someway to active auto config to suscription yet created?