Openssl1.1.1o checking hostname fails on DNS SAN wildcard

On a Centos 7 machine, upgrading from Python 3.6.8 to python 3.10.4 and upgrading openssl 1.0.2-fips to openssl 1.1.1o, I’m having an issue getting openssl to verify the hostname for a DNS wildcard SAN in the certificate for our mutliple kafka brokers (kafka-0, kafka-1, or kafka-2).

I know I could get around this issue by updating our kafkaAdminClient configs to have ssl_check_hostname set to False, but that isn’t an ideal solution security-wise.

It seems like the ssl.wrap_socket() function is not reading the DNS wildcard properly.

—test code—

import ssl, socket, pprint

context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
context.verify_mode = ssl.CERT_REQUIRED
context.load_default_certs()
s = socket.socket(socket.AF_INET)
ssl_sock = context.wrap_socket(s, server_hostname="kafka")
ssl_sock.connect(('kafka', 9092))
cert = ssl_sock.getpeercert()
pprint.pprint(cert)

—output—

$ python testcert.py

{'issuer': ((('commonName', 'kubernetes'),),),
 'notAfter': 'Feb 24 22:56:26 2023 GMT',
 'notBefore': 'Feb 24 22:56:26 2022 GMT',
 'serialNumber': '<redacted>',
 'subject': ((('commonName', 'kafka'),),),
 'subjectAltName': (('DNS', 'kafka'), ('DNS', '*.kafka')),
 'version': 3}

—Certificate information:—

---
Can't use SSL_get_servername
depth=1 CN = kubernetes
verify return:1
depth=0 CN = kafka
verify return:1
---
Certificate chain
 0 s:CN = kafka
   i:CN = kubernetes
 1 s:CN = kubernetes
   i:CN = kubernetes
----
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
----
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
---
$ openssl s_client -connect kafka:9092 | openssl x509 -noout -text | grep DNS:
Can't use SSL_get_servername
depth=1 CN = kubernetes
verify return:1
depth=0 CN = kafka
verify return:1
                DNS:kafka, DNS:*.kafka

—changing test script to:—

ssl_sock = context.wrap_socket(s, server_hostname="kafka-1.kafka")
ssl_sock.connect(('kafka-1.kafka', 9092))

—output—

$ python testcert.py
Traceback (most recent call last):
  File "/usr/src/testcert.py", line 10, in <module>
    ssl_sock.connect(('kafka-1.kafka', 9092))
  File "/usr/local/lib/python3.10/ssl.py", line 1374, in connect
    self._real_connect(addr, False)
  File "/usr/local/lib/python3.10/ssl.py", line 1365, in _real_connect
    self.do_handshake()
  File "/usr/local/lib/python3.10/ssl.py", line 1341, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for 'kafka-1.kafka'. (_ssl.c:997)

—changing test script to:—

ssl_sock = context.wrap_socket(s, server_hostname='kafka-1')
ssl_sock.connect(('kafka-1', 9092))

—output—

$python testcert.py
Traceback (most recent call last):
  File "/usr/src/testcert.py", line 10, in <module>
    ssl_sock.connect(('kafka-1', 9092))
  File "/usr/local/lib/python3.10/ssl.py", line 1374, in connect
    self._real_connect(addr, False)
  File "/usr/local/lib/python3.10/ssl.py", line 1361, in _real_connect
    super().connect(addr)
socket.gaierror: [Errno -2] Name or service not known

----python build & system info—
Our manual installation steps of openssl 1.1.1o and python 3.10.4 from Dockerfile:

RUN yum clean all && yum makecache && yum -y update \
    && yum install -y make gcc perl-core pcre-devel wget zlib-devel \
    && wget https://ftp.openssl.org/source/openssl-1.1.1o.tar.gz \
    && tar -xf openssl-1.1.1o.tar.gz \
    && cd openssl-1.1.1o \
    && ./config --prefix=/usr --openssldir=/etc/pki/tls --libdir=lib no-shared zlib-dynamic \
    && make && make install \
    && cd ../ \
    && rm -rf openssl-1.1.1o.tar.gz

# Installing python3 and pip
ARG PYTHON_FULL_VERSION (= 3.10.4)
ARG PYTHON_HASH
ARG PYTHON_VERSION (= 3.10)
RUN yum -y install bzip2-devel libffi-devel xz-devel \
    && TAR_FILE=Python-${PYTHON_FULL_VERSION}.tgz \
    && wget https://www.python.org/ftp/python/${PYTHON_FULL_VERSION}/${TAR_FILE} \
    && echo "${PYTHON_HASH}  ${TAR_FILE}" | md5sum -c - \
    && tar xvf ${TAR_FILE} \
    && cd Python-*/ \
    && ./configure --with-openssl=/usr/src/openssl-1.1.1o --enable-optimizations \
    && make altinstall \
    && cd .. \
    && rm -rf Python-*
$ python --version
Python 3.10.4
$ openssl version -a
OpenSSL 1.1.1o  3 May 2022
built on: Mon May 16 18:11:42 2022 UTC
platform: linux-x86_64
options:  bn(64,64) rc4(8x,int) des(int) idea(int) blowfish(ptr)
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -O3 -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPOLY1305_ASM -DZLIB -DZLIB_SHARED -DNDEBUG
OPENSSLDIR: "/etc/pki/tls"
ENGINESDIR: "/usr/lib/engines-1.1"
Seeding source: os-specific

—centos 7 details—

cat /etc/centos-release
CentOS Linux release 7.9.2009 (Core)

uname -a
Linux 4.19.0-18-cloud-amd64 #1 SMP Debian 4.19.208-1 (2021-09-29) x86_64 x86_64 x86_64 GNU/Linux

Not sure if I should open up a bug ticket. Thank you for any help.

OpenSSL may not like the fact that your hostname has only two labels and the first label is a wildcard. Typically a hostname has a TLD, domain, and additional labels, e.g. kafka-1.kafka.internal for *.kafka.internal. What is the output of

openssl s_client -connect kafka:9092 -verify_hostname kafka-1.kafka -servername kafka-1.kafka

?

Thank you for the quick reply!

Hmm, yeah, looks like the output of the above openssl command is also a hostname mismatch:

$ openssl s_client -connect kafka:9092 -verify_hostname kafka-1.kafka -servername kafka-1.kafka
CONNECTED(00000003)
depth=0 CN = kafka
verify error:num=62:Hostname mismatch
verify return:1
depth=1 CN = kubernetes
verify return:1
depth=0 CN = kafka
verify return:1
---
Certificate chain
 0 s:CN = kafka
   i:CN = kubernetes
 1 s:CN = kubernetes
   i:CN = kubernetes
---
-----END CERTIFICATE-----
subject=CN = kafka

issuer=CN = kubernetes

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1935 bytes and written 408 bytes
Verification error: Hostname mismatch
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
etc