Just did a quick test, the certs do not bind to the port, only the domain/fqdn. So in short your reverse proxy/application is doing something wrong. Do you have the cert files? Can you test them inside a ubuntu:24.04 docker with the script bellow? (you’ll need to copy two cert files). That does TLS and is the application all in one script, but it could be two scripts one acting as the reverse proxy or whatever, doesn’t make a difference from the point of view of the client.

Lets Encrypt doesn’t do anything in port 80/443 unless you’re using the http challlenge AFAIK. And once you have the certs, they aren’t really involved in the connection, thus that can’t be the issue. Test by using curl against the script below, or your own infrastructure (each step/chain of it, the reverse proxy, the application ip, etc.)

But in short I think your reverse proxy configuration is just wrong, or you’re accessing it the wrong way on the client side. For example, using https://example.com instead of https://example.com:5050.

# docker run --rm --net host -it ubuntu:24.04
# then install python3 and run this
import http.server
import ssl

PORT = 5201  # Change to your desired port
CERT_FILE = "/fullchain.pem"  # Path to your certificate file
KEY_FILE = "/key.pem"    # Path to your private key file

# Create a basic HTTP request handler
class SimpleHTTPRequestHandler(http.server.SimpleHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200)
        self.send_header("Content-type", "text/html")
        self.end_headers()
        self.wfile.write(b"<h1>Welcome to the secure static server!</h1>")

# Set up the HTTP server
httpd = http.server.HTTPServer(("0.0.0.0", PORT), SimpleHTTPRequestHandler)

# Set up SSL context
ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
ssl_context.load_cert_chain(certfile=CERT_FILE, keyfile=KEY_FILE)

# Wrap the server socket with SSL
httpd.socket = ssl_context.wrap_socket(httpd.socket, server_side=True)

print(f"Serving HTTPS on port {PORT}")
httpd.serve_forever()

I recommend watching this video: https://www.youtube.com/watch?v=qbKGw8MQ0i8