Issue Nginx configuration to accept websocket requests (SSL/WSS)

[AhmedEx]

Server operating system version

Ubuntu 22.04.1

Plesk version and microupdate number

18.0.46

i have plesk server with apache2 and nginx and i have a websocket app it works on port

2589

and i used ratchet and autobahn to work with websocket, nginx version

nginx version: nginx/1.20.2

and ufw status

Status: inactive

and my nginx directive config for websocket

location ^~ /WsConn
{
    add_header 'Access-Control-Allow-Origin' '*';

    proxy_pass             http://localhost:2589;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Client-Verify SUCCESS;
    proxy_set_header X-NginX-Proxy true;
    proxy_read_timeout     60;
    proxy_connect_timeout  60;
    proxy_redirect         off;

    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
}

my js to connect to wss

new ab.Session(
    'wss://domain.com/WsConn',
function() {
    console.log('Connected!');
}
function() {
    console.warn('SOCKET DISCONNECTED, WAIT NOW!!');
}, {
    'skipSubprotocolCheck': true
}

and i keep getting connection failed in my browser

btw it was works so fine on my old VPS and there was some settings inside the directive config for nginx that made it worked, but when i moved on new VPS i lost this settings, so i hope if someone can help me setup the perfect settings for it again to make it accept websocket requests.

EDIT: proxy mode is disabled, and still dun work.

thanks.

 

[AhmedEx]

i can't edit the main thread.
websocket already running and works fine in server, so it's sure about nginx configuration to process the requests to websocket.

 

[AhmedEx]

i found this errors in proxy_error.log

2022/08/23 19:25:17 [error] 175835#0: *2873868 upstream prematurely closed connection while reading response header from upstream, client: xxx.xxx.xxx.xxx, server: domain.com, request: "GET /WsConn?sessid=mrgk7rg627morchjqml982iug2&&tk=8f45d30d6899f54d8f6c54886cd00286&&uid=1&&isios=0 HTTP/1.1", upstream: "http://127.0.0.1:2589/WsConn?sessid...5d30d6899f54d8f6c54886cd00286&&uid=1&&isios=0", host: "www.domain.com"
2022/08/23 19:25:17 [error] 175835#0: *2873868 connect() failed (111: Connection refused) while connecting to upstream, client: xxx.xxx.xxx.xxx, server: domain.com, request: "GET /WsConn?sessid=mrgk7rg627morchjqml982iug2&&tk=8f45d30d6899f54d8f6c54886cd00286&&uid=1&&isios=0 HTTP/1.1", upstream: "http://[::1]:2589/WsConn?sessid=mrgk7rg627morchjqml982iug2&&tk=8f45d30d6899f54d8f6c54886cd00286&&uid=1&&isios=0", host: "www.domain.com"

 

[deno]

Hi AhmedEx,

I had the same issue with Plesk & Docker and found a solution that may help you as well.

For me, setting up the nginx directive config in Plesk didn't help so you could try following:

1. Go to the Nginx directive Config in your domain and delete the entries under "Websites & Domains - Hosting DNS - Apache & nginx"
2. Make sure the port mapping is set correctly in Docker Settings and "Websites & Domains - Dashboard - Docker Proxy Rules" and remember the port number
3. Now you have to manually add the following lines to the nginx.conf located at /var/www/vhosts/system/<yourdomainname>/conf/nginx.conf

proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_http_version 1.1;

The Proxy Pass Port (in this case 32777) should be the same as the Port value found in step 2. When you are finished, it should look like this:

        #extension docker begin
        location ~ ^/.* {
                proxy_pass http://0.0.0.0:32777;
                proxy_set_header        Upgrade $http_upgrade;
                proxy_set_header        Connection "upgrade";
                proxy_http_version       1.1;
                proxy_set_header Host              $host;
                proxy_set_header X-Real-IP         $remote_addr;
                proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;
        }

        #extension docker end

4. Save your changes and restart nginx (via SSH sudo systemctl restart nginx or directly in Plesk "Tools & Settings - Services Management - Reverse Proxy Server (nginx)"

NOTE: The nginx.conf will be overwritten if you make any changes to your configuration at "Websites & Domains - Hosting DNS - Apache & nginx", so you will need to add the changes manuallly again.

I hope this helps.

 

[Kaspar]

@deno thank you for sharing your solution. I am curious to know why adding this directive in Plesk does not work for you? Because it should. Do you encounter any error when trying?

 

[deno]

@Kaspar I really don't know, I can't find any errors until I open an URL and then get the websocket error.

It's not that i can't add the directive, I can save it and it appears in the vhost_nginx.conf. I'm really no expert, but somehow it seems that the directives in vhost_nginx.conf are not prioritized or correctly included. I spent days finding a solution and tried everything, but this is the only thing that worked for me and it seems I'm not alone.

 

[Kaspar]

Thank you for posting back. It could well be that directives in vhost_nginx.conf are lower prioritized or perhaps even conflict with other directives. Could you perhaps share the full content of your domain's nginx.conf file? Obscuring your domain of course. I am curious to see if there perhaps is something at Plesk's side that's blocking proper usage of websocket.

 

[deno]

@Kaspar of course, thanks for your efforts. Here you go:

#ATTENTION!
#
#DO NOT MODIFY THIS FILE BECAUSE IT WAS GENERATED AUTOMATICALLY,
#SO ALL YOUR CHANGES WILL BE LOST THE NEXT TIME THE FILE IS GENERATED.

server {
    listen 12.34.56.789:443 ssl http2;

    server_name subdomain.domain.com;
    server_name www.subdomain.domain.com;
    server_name ipv4.subdomain.domain.com;

    ssl_certificate             /opt/psa/var/certificates/scfRN8Jos;
    ssl_certificate_key         /opt/psa/var/certificates/scfRN8Jos;

    client_max_body_size 134217728;

    access_log "/var/www/vhosts/system/subdomain.domain.com/logs/proxy_access_ssl_log";
    error_log "/var/www/vhosts/system/subdomain.domain.com/logs/proxy_error_log";

    root "/var/www/vhosts/nowrds.com/subdomain.domain.com";

    #extension sslit begin

    #extension sslit end

    #extension letsencrypt begin
    location ^~ /.well-known/acme-challenge/ {
        root /var/www/vhosts/default/htdocs;

        types { }
        default_type text/plain;

        satisfy any;
        auth_basic off;
        allow all;

        location ~ ^/\.well-known/acme-challenge.*/\. {
            deny all;
        }
    }
    #extension letsencrypt end

    #extension sectigo begin
    location ^~ /.well-known/pki-validation/fileauth.txt {
        try_files $uri $uri/ =404;
    }

    location ^~ /.well-known/pki-validation/ {
        root /var/www/vhosts/default/htdocs;

        types { }
        default_type text/plain;

        satisfy any;
        auth_basic off;
        allow all;

        location ~ ^/\.well-known/pki-validation.*/\. {
            deny all;
        }
    }
    #extension sectigo end

    #extension performance-booster begin
    # Additional directives added by Plesk Optimization Settings

    # Additional directives added by Plesk Optimization Settings
    #extension performance-booster end

    #extension docker begin
    location ~ ^/.* {
        proxy_pass http://0.0.0.0:32456;
        proxy_set_header        Upgrade $http_upgrade;
                proxy_set_header        Connection "upgrade";
                proxy_http_version       1.1;
        proxy_set_header Host              $host;
        proxy_set_header X-Real-IP         $remote_addr;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    #extension docker end

    error_page 400 "/error_docs/bad_request.html";
    error_page 401 "/error_docs/unauthorized.html";
    error_page 403 "/error_docs/forbidden.html";
    error_page 404 "/error_docs/not_found.html";
    error_page 500 "/error_docs/internal_server_error.html";
    error_page 405 "/error_docs/method_not_allowed.html";
    error_page 406 "/error_docs/not_acceptable.html";
    error_page 407 "/error_docs/proxy_authentication_required.html";
    error_page 412 "/error_docs/precondition_failed.html";
    error_page 414 "/error_docs/request_uri_too_long.html";
    error_page 415 "/error_docs/unsupported_media_type.html";
    error_page 501 "/error_docs/not_implemented.html";
    error_page 502 "/error_docs/bad_gateway.html";
    error_page 503 "/error_docs/maintenance.html";

    location ^~ /error_docs {
        root "/var/www/vhosts/nowrds.com";
    }

    location ~ /\.ht {
        deny all;
    }

    location ~ ^/(plesk-stat|awstats-icon|webstat|webstat-ssl|ftpstat|anon_ftpstat) {
        auth_basic "Domain statistics";
        auth_basic_user_file "/var/www/vhosts/system/subdomain.domain.com/pd/d..httpdocs@plesk-stat";
        autoindex on;

        location ~ ^/plesk-stat(.*) {
            alias /var/www/vhosts/system/subdomain.domain.com/statistics/$1;
        }

        location ~ ^/awstats-icon(.*) {
            alias /usr/share/awstats/icon/$1;
        }

        location ~ ^/(.*) {
            alias /var/www/vhosts/system/subdomain.domain.com/statistics/$1;
        }
    }

    location ~ ^/~(.+?)(/.*?\.php)(/.*)?$ {
        fastcgi_read_timeout 600;
        alias /var/www/vhosts/nowrds.com/web_users/$1/$2;
        fastcgi_split_path_info ^((?U).+\.php)(/?.+)$;
        try_files $uri $fastcgi_script_name =404;
        fastcgi_param PATH_INFO $fastcgi_path_info;
        fastcgi_pass "unix:/var/www/vhosts/system/subdomain.domain.com/php-fpm.sock";
        include /etc/nginx/fastcgi.conf;

    }

    location ~ \.php(/.*)?$ {
        fastcgi_read_timeout 600;
        fastcgi_split_path_info ^((?U).+\.php)(/?.+)$;
        try_files $uri $fastcgi_script_name =404;
        fastcgi_param PATH_INFO $fastcgi_path_info;
        fastcgi_pass "unix:/var/www/vhosts/system/subdomain.domain.com/php-fpm.sock";
        include /etc/nginx/fastcgi.conf;

    }

    index "index.html" "index.cgi" "index.pl" "index.php" "index.xhtml" "index.htm" "index.shtml";

    disable_symlinks if_not_owner "from=/var/www/vhosts/nowrds.com";

    add_header X-Powered-By PleskLin;

    include "/var/www/vhosts/system/subdomain.domain.com/conf/vhost_nginx.conf";
}

server {
    listen 12.34.56.789:80;

    server_name subdomain.domain.com;
    server_name www.subdomain.domain.com;
    server_name ipv4.subdomain.domain.com;

    client_max_body_size 134217728;

    access_log "/var/www/vhosts/system/subdomain.domain.com/logs/proxy_access_log";
    error_log "/var/www/vhosts/system/subdomain.domain.com/logs/proxy_error_log";

    location / {
        return 301 https://$host$request_uri;
    }
}
$

 

[Kaspar]

@deno, thank you. There doesn't seems to be anything out of the ordinary in your nginx.conf file as far as I can tell. What image docker are you running in Docker?

 

[deno]

@Kaspar I'm running Baserow and NocoDB so far, will try a few others as well, but the issue should not be related to the images as far as i can tell.

 

[Kaspar]

You're absolutely right, it should matter which images your using. I was just curious and wanted to know so I could see if I could replicate the issue with the same images.

If did however recently performed some tests with Baserow based on posts from another forum topic abount websockets. I didn't encounter any issues with websockets in Baserow when adding the Nginx location blocks to the additional Nginx directives in Plesk. I got the directives from the Baserow documentation.

Is there any specific reason you used the ~ ^/.* modifier instead of the directives from the Baserow documentation? No judgment, just asking out of curiosity.

 

[deno]

I participated in the same forum topic before because I encountered this issue. Tried everything multiple times, even the setup you mentioned, nothing worked, until I manually added the directives directly to the config. And no, there is no specific reason for it, I didn't change the modifier created by docker, just added the 3 lines. Having spent days with the problem, I had no intention of changing anything else as it finally worked that way. Do you see any disadvantage in that?

 

[Kaspar]

I participated in the same forum topic before because I encountered this issue. Tried everything multiple times, even the setup you mentioned, nothing worked, until I manually added the directives directly to the config. And no, there is no specific reason for it, I didn't change the modifier created by docker, just added the 3 lines. Having spent days with the problem, I had no intention of changing anything else as it finally worked that way. Do you see any disadvantage in that?

@deno thank you! Reading your post made me realize what actually happend and what causes the issue here.

So, when creating Docker Proxy Rules for a domain (as you mentioned in step 2 of your first post in this topic) these proxy rules get added to the nginx.conf. No surprise here. However these proxy rules take precedence over any other directives/rules added to the additional nginx directives of the domain. Hence, no matter what you add to the additional nginx directives, websockets will (unfortunately) never work.

The solution would to remove the Docker Proxy Rules from the domain, and instead add the needed proxy directives manually to the additional nginx directives. I hope this makes sense?

I've created a feature suggestion for the use of websockets when creating docker proxy rules. I encourage anyone reading who's also using docker to vote for this feature.

 

[deno]

@Kaspar that totally makes sense now, I checked it on my end and as you say the docker proxy rules take precedence over any other directives/rules, so you can't use both at the same time.
It would have been great to have at least some kind of notification or alert that those directives won't working as long as docker proxy rules are set.

On my end I copied the exact rules from the nginx.conf, deleted the docker proxy rules and set the nginx directives and now websockets work. However, unlike in many examples, I could not use the domain as a value for the proxy_pass and had to use http://0.0.0.0:<PORT>

Your suggestion makes perfect sense, I have already voted for it and hope they implement it soon as it will make things much easier and avoid misunderstandings.

Thanks for your efforts and clarification!