I’m an engineer at AWS working on AWS-LC, AWS’s open-source fork of Google’s BoringSSL. We have patch sets to CPython 3.10 through tip-of-main that build CPython’s ssl and hashlib modules against AWS-LC’s libssl and libcrypto respectively. We are committed to backwards compatibility and have CI jobs (here and here) asserting every change’s compatibility with many different open-source projects. We use these tests to catch compatibility regressions before they’re merged, and we plan to add a CPython compatibility test to this suite.
AWS-LC has recently made a number of CPU-specific performance optimizations for AWS Graviton 2, AWS Graviton 3, and Intel x86-64 with AVX-512 instructions. We’ve formally verified a subset of AWS-LC’s cryptographic primitives, and continue to invest in expanding this coverage. To support meeting FIPS 140-3 compliance requirements, AWS-LC can be built in FIPS mode. To give CPython users a well-documented and supported way to take advantage of these investments in performance, correctness, and compliance, we would like to upstream support for AWS-LC into mainline CPython. We believe that this would provide the best experience for users wishing to build CPython against AWS-LC. It would also allow users to skip the (often brittle) process of maintaining and applying their own patch sets to build CPython with AWS-LC.
The proposed integration would support all features of CPython’s ssl and hashlib modules except for post-handshake authentication in TLSv1.3, as AWS-LC does not yet support this. In our fork of CPython we have a unit test asserting that CPython built against AWS-LC, lacking post-handshake authentication, behaves as expected. The only other feature gap we’ve identified is AWS-LC’s lack of support for the blake2s hash function, where we suggest using CPython’s own implementation of blake2. If you folks agree that this integration would be useful for upstream CPython, I’d be happy to put together a PR.
I don’t think we have BoringSSL support in Python to begin with because the BoringSSL team warned us not to due to the lack of API stability promises. As such, I would guess AWS-LC would have the same lack of stability, correct?
No, we’re aiming to keep AWS-LC’s API as stable as we can through time. AWS-LC is indeed a fork of BoringSSL, but within AWS-LC’s libssl, we are focusing specifically on OpenSSL API compatibility (and not ABI compatibility). We plan to add per-change checks to AWS-LC’s CI for each open source project we integrate with, including CPython. This will prevent us from merging any changes that break compatibility with supported versions of CPython, and also alert us to any breaking changes made upstream in CPython so we can send PRs.
It seems no one from the core team is interested in supporting this.
Most of the changes in your branch, by volume, are differences in expected exception text in the test suite. I’d be OK merging some of those – especially in cases where the BoringSLL text is clearer or equivalent. (If there’s important information in those messages, it should IMO be exposed in a structured way.)
IMO we could get to the point where you can get CPython to compile with AWS-LC without special-casing it (i.e. no Py_OPENSSL_IS_AWSLC), but relevant test cases still fail.
You’ll still need a patch, but a more focused one.
Thanks folks. I’ll draft a PR modifying tests to make them more flexible around error text differences and post that here for further discussion when it’s ready for review.
I’ve published a PR here relaxing error message expectations across a number of tests.This cuts our CPython main branch diff down to 170 lines, almost all related to PHA support.
Ok, I’ve posted another PR allowing for builds against crypto libraries that don’t support the BLAKE2 hash function. @gpshead has expanded this PR to detect lack of support for other hash functions as well.
There is one additional PR I would like to post for detecting lack of support for post-handshake authentication, as AWS-LC doesn’t currently plan to implement post-handshake authentication. The change would look something like this.
With that change, AWS-LC would be able to build against CPython with no source code patches, and only fail expected tests due to lack of certain features.
Ok, I’ve posted the last change in the series here. This change looks for a configuration macro OPENSSL_NO_TLS_PHA that is now defined by AWS-LC, and may be defined by other crypto libraries lacking support for post-handshake authentication (PHA).
Thanks folks! The last PR in the series has been merged. With that change, CPython main (3.14 pre-release) now builds against AWS-LC without any source modification. AWS-LC has a CI integration test to prevent compatibility regressions on our end. If the CPython maintainer team would like a similar test in CPython’s CI, I can put together a PR testing AWS-LC in the same way CPython currently tests against OpenSSL.
I built cpython 3.14/main with the patches from here. However, attempting to make install causes ensurepip to error with “No cipher can be selected”… Any ideas?
Thanks for your interest in working with Python + AWS-LC! The patch file you linked should be sufficient for building current CPython against AWS-LC, but does require some additional ./configure parameters (see here).
If applying those build configuration parameters doesn’t resolve your issue, please file a GitHub Issue here with terminal commands + output and tag me @WillChilds-Klein.
