Python and OpenSSL FIPS mode

[stratakis (Charis)]

Hi folks,

I’d like to start a discussion regarding CPython’s compatibility with OpenSSL’s FIPS mode.

FIPS-140 is a set of security standards for cryptographic modules, mainly used in US Federal agencies, contractors and vendors.

FIPS sets various restrictions in regards to the permissible digests, key sizes etc, with one of the prominent one is disabling the md5 algorithm, and it’s implemented system-wide. Some more information is provided here.

Any developer, developing or deploying applications on a FIPS enforced environment needs to be using FIPS compliant software. While I’m not advocating for fully supporting an upstream FIPS compliant codebase, I’d like to work on a best-effort initiative, as the required changes, I believe, are not that intrusive and do not require significant maintenance.

Motivation is coming from maintaing a patch downstream mainly for Red Hat Enterprise Linux to provide said compatibility. We’d like to share that patch with others.

Currently FIPS mode can be determined through _hashlib.get_fips_mode() which was added in Python 3.9, along with various fixes around that.

Now I’d like to move more of the bits of our downstream patch upstream with the main point being making CPython more compilable and usable for system utilizing OpenSSL’s FIPS mode.

On a high level, the patch:

• exposes blake2b and blake2s from OpenSSL (although they are limited in their interface, and using them for security is not FIPS compliant),
• disables Python’s blake2 interfaces in case we are in FIPS mode
• disables the hash-based .pyc validation in FIPS mode, due to the non-fips-compliant sip24 digest
• adds various test fixes.

The patch is assuming that Python has been compiled with --with-builtin-hashlib-hashes=blake2 only and it needs to be generalized for all the other cases, main one being disabling falling back to the internal implementations in case they are built if we are under FIPS.

I’m also already providing, for some time now, two buildbots compiling Python under FIPS enabled systems:

• Buildbot
• Buildbot

[stratakis (Charis)]

In the hopes of the issue getting more traction I’m cc’ing some folks who might be interested into that or could provide some more feedback.

@tiran @encukou @vstinner @gpshead

[gpshead (Gregory P. Smith)]

Initial thoughts on the patches themselves:

@stratakis

On a high level, the patch:

• exposes blake2b and blake2s from OpenSSL (although they are limited in their interface, and using them for security is not FIPS compliant),
• disables Python’s blake2 interfaces in case we are in FIPS mode

The OpenSSL ones don’t provide the complete API necessary to be hashlib compatible so we should not use them - I don’t want to see new code that doesn’t support the entire API added to CPython.

Just have them raise an error if usedforsecurity=True when in “FIPS mode”. There is no added value in exposing them from OpenSSL builds that oddly support them.

@stratakis

disables the hash-based .pyc validation in FIPS mode, due to the non-fips-compliant sip24 digest

I don’t like this one. Hash-based .pyc validation is not a security feature. The fast hash is not being used for security. Disabling it just hurts the rare-ish users who actually use validated hash-based .pyc files (it isn’t the default) in their deployments. If you are disabling just this, you really should disable all use of .pyc files entirely. A timestamp + size comparison is just as much a not-for-security hash as sip24 is. It’s just that nobody put “hash” in the name of that comparison.

@stratakis

The patch is assuming that Python has been compiled with --with-builtin-hashlib-hashes=blake2 only and it needs to be generalized for all the other cases, main one being disabling falling back to the internal implementations in case they are built if we are under FIPS.

hashlib is moving in the direction of potentially not using OpenSSL at all in the future in favor of only supporting the new HACL* based built-ins. If we do get to that point, keeping _hashopenssl.c around won’t make a lot of sense…

[gpshead (Gregory P. Smith)]

This patch set seems to be going beyond OpenSSL by trying to prevent things not in OpenSSL from doing their job. At that point it ceases to really be “OpenSSL FIPS mode” and turns into a feature “CPython FIPS mode sponsored by $Vendors X,Y,&Z”. Which is plausible as a feature request but needs an explicit definition of scope and buildbot(s) that explicitly tests such a build and environment validating that things that should not work do not, and things that should do. With engaged contributors who’ll work to keep that stable as the codebase evolves.

It isn’t the kind of thing that the open source community is likely interested in maintaining on their own (it’s someones interpretation of one government’s bureaucracy). So a goal I’d recommend is to make understanding of the scope and maintenance of code that dances around such a self-limiting feature as easy as possible. Most people never want to think about it at all. All of the existing OpenSSL FIPS mode support within CPython today came from people specifically employed to make it happen, RedHat primarily.

Note that the HACL* hash implementations are also in the formally verified camp, just not by the US Government’s National Institute of Standards and Technology. Someone $enterprising could presumably make that happen so that they get a US NIST seal of approval similar to the one the OpenSSL fips module claims to have - assuming that is what this is really all about.

[encukou (Petr Viktorin)]

I’m quite biased; I don’t think I should be making decisions for CPython in this area.

But for the record, I think the current state draws a reasonable boundary between what CPython should do and what should be the job of a redistributor (who can both ensure OpenSSL is available, and enact and test system-wide restrictions).

[vstinner (Victor Stinner)]

@gpshead

This patch set seems to be going beyond OpenSSL by trying to prevent things not in OpenSSL from doing their job. At that point it ceases to really be “OpenSSL FIPS mode” and turns into a feature “CPython FIPS mode sponsored by $Vendors X,Y,&Z”. Which is plausible as a feature request but needs an explicit definition of scope and buildbot(s) that explicitly tests such a build and environment validating that things that should not work do not, and things that should do. With engaged contributors who’ll work to keep that stable as the codebase evolves.

I’m working for Red Hat with @stratakis and I’m fine to care of maintaining these FIPS changes upstream. We are already maintaining these changes downstream for years. The idea is to make it available to other users who have to respect FIPS constraints.

If tomorrow, hashlib gets rid of OpenSSL, that would be great, and we can adapt the FIPS patch then.

[stratakis (Charis)]

@gpshead

The OpenSSL ones don’t provide the complete API necessary to be hashlib compatible so we should not use them - I don’t want to see new code that doesn’t support the entire API added to CPython.

Just have them raise an error if usedforsecurity=True when in “FIPS mode”. There is no added value in exposing them from OpenSSL builds that oddly support them.

I agree that exposing them from OpenSSL is undesirable as they lack features. Raising an error sounds reasonable here. On a semi-relevant note here, compiling Python without the internal blake2 implementation results in various test failures.

@gpshead

I don’t like this one. Hash-based .pyc validation is not a security feature. The fast hash is not being used for security. Disabling it just hurts the rare-ish users who actually use validated hash-based .pyc files (it isn’t the default) in their deployments. If you are disabling just this, you really should disable all use of .pyc files entirely. A timestamp + size comparison is just as much a not-for-security hash as sip24 is. It’s just that nobody put “hash” in the name of that comparison.

I’m in the process of the removing this part of the patch as it was an overkill and siphash is not used in a security context here.

@gpshead

hashlib is moving in the direction of potentially not using OpenSSL at all in the future in favor of only supporting the new HACL* based built-ins . If we do get to that point, keeping _hashopenssl.c around won’t make a lot of sense…

@gpshead

Note that the HACL* hash implementations are also in the formally verified camp, just not by the US Government’s National Institute of Standards and Technology. Someone $enterprising could presumably make that happen so that they get a US NIST seal of approval similar to the one the OpenSSL fips module claims to have - assuming that is what this is really all about.

Reaching that point is a long-term goal and replacing the internal digests implementation is a different thing than stopping supporting openssl or other cryptographic libraries. And if that point is reached the discussion will be entirely different in scope.

@gpshead

This patch set seems to be going beyond OpenSSL by trying to prevent things not in OpenSSL from doing their job. At that point it ceases to really be “OpenSSL FIPS mode” and turns into a feature “CPython FIPS mode sponsored by $Vendors X,Y,&Z”. Which is plausible as a feature request but needs an explicit definition of scope and buildbot(s) that explicitly tests such a build and environment validating that things that should not work do not, and things that should do. With engaged contributors who’ll work to keep that stable as the codebase evolves.

As noted in the initial post, I’m already providing 2 buildbots that test the FIPS mode functionality. And I’m quite well versed in this part of the codebase and as mentioned by @vstinner we’ve been maintaining those patches for years. With the buildbot testing I don’t think it’s gonna be a big maintenance trouble.

@gpshead

It isn’t the kind of thing that the open source community is likely interested in maintaining on their own (it’s someones interpretation of one government’s bureaucracy). So a goal I’d recommend is to make understanding of the scope and maintenance of code that dances around such a self-limiting feature as easy as possible. Most people never want to think about it at all. All of the existing OpenSSL FIPS mode support within CPython today came from people specifically employed to make it happen, RedHat primarily.

Would starting with a PR with (almost) all the changes incorporated to provide a better overview of the scope make sense?