Building with OpenSSL 3.0 for the GitHub Release binaries

[Edward-Knight]

OpenSSL 1.1.1 is going EOL in September, and it looks like the builds here are currently using 1.1.1s.
From a quick glance it seems like CPython supports building with OpenSSL 3.0 from 3.9 onwards.

I was wondering if there were any plans about upgrading? I imagine Python 3.8 will still need to use OpenSSL 1.1.1, but I believe the other builds can be upgraded. I can help with an MR if there isn't anything blocking.

[indygreg]

We should ideally migrate to OpenSSL 3 for every CPython version that supports it. This has been on my mind for a while I just haven't done it yet.

On Windows, we might want to maintain parity with whatever CPython is shipping. i.e. we'll want to wait on upstream to ship a CPython 3.9 using OpenSSL before we do. (I anticipate CPython will run into issues getting OpenSSL 3 working right on Windows because pretty much nobody builds CPython from source on Windows and the build configurations that aren't identical to the official CPython releases aren't well tested.) But if CPython does have OpenSSL 3 test coverage, by all means upgrade.

Patches would be very much appreciated!

[Edward-Knight]

From poking around in the CPython GitHub Actions build.yml and PCBuild/get_externals.bat it seems that they currently build using these versions of OpenSSL:

	Linux SSL tests	Linux build	macOS build	macOS installer	Windows build
Python 3.8	1.0.2u, 1.1.1u, 3.0.9, 3.1.1	1.1.1u	1.1 (PR for 3.0)	1.1.1u	1.1.1u
Python 3.9	1.0.2u, 1.1.0l, 1.1.1u, 3.0.9, 3.1.1	1.1.1u	1.1 (PR for 3.0)	1.1.1u	1.1.1u
Python 3.10	1.1.1u, 3.0.9, 3.1.1	1.1.1u	1.1 (PR for 3.0)	1.1.1u	1.1.1u
Python 3.11	1.1.1u, 3.0.9, 3.1.1	1.1.1u	3.0	1.1.1u	1.1.1u

It looks like OpenSSL 3.0 should work on Python 3.8. The ticket I linked in my original post does have some PRs into 3.8, and although OpenSSL 3.0 tests for 3.8 were removed in python/cpython@438f8cd, it looks like they were added back in in python/cpython@ddd495e.

---

@indygreg would you be okay with a PR that:

• Builds Python 3.8–3.11 for Linux and macOS with OpenSSL 3.0.9 (currently 1.1.1s)
• Builds Python 3.8–3.11 for Windows with OpenSSL 1.1.u (currently 1.1.1s)

[erlend-aasland]

FTR / FWIW, there are three open CPython PRs for running the macOS CI with OpenSSL 3.0:

• [3.10] CI: Bump macOS build to use OpenSSL v3.0 (GH-105538) python/cpython#105869
• [3.9] CI: Bump macOS build to use OpenSSL v3.0 (GH-105538) python/cpython#105871
• [3.8] CI: Bump macOS build to use OpenSSL v3.0 (GH-105538) python/cpython#105872

(These PRs are awaiting approval from the respective release managers, since merging to security fix branches is constrained.)

The macOS CI for 3.11, 3.12 and 3.13 (main) are already using OpenSSL 3.0. The devguide also recommends OpenSSL 3.0.

See also:

• Use OpenSSL 3.0.x in our binary builds python/cpython#99079

Quoting python/cpython#99079 (comment):

If we assume that that we should focus on 3.0.x support on macOS for the time being at least, I think the first step would be to update our GitHub Workflow macOS CI jobs for 3.11+ to use 3.0.x from Homebrew: currently they are using 1.1.x. We should also update the devguide recommendations for macOS to use 3.0.x; there is a python/devguide#1058 that should be changed to use 3.0.x. Also, any macOS buildbots should be updated, as feasible. With that additional experience we can then move the macOS installers to using 3.0.x sometime in the not-too-distant future.

[erlend-aasland]

Also, the macOS installer uses the following:

https://github.com/python/cpython/blob/04492cbc9aa45ac2c12d22083c406a0364c39f5b/Mac/BuildScript/build-installer.py#L243-L256

[Edward-Knight]

Thanks Erlend I'll update my table to include that

[Edward-Knight]

OpenSSL has changed license in 3.0 to Apache License 2.0, which I assume isn't an issue but I'll note it here as it will require an update to the license info we bundle: https://www.openssl.org/source/license.html

[erlend-aasland]

FYI, OpenSSL 3.0.x is now used in the official binary builds: python/cpython#99079 (comment)

[Edward-Knight]

Thanks Erlend 👍

[indygreg]

I think I'm pretty close to having OpenSSL 3.0 working. Just need to figure out how to handle a new dependency on libatomic on Linux.

On Linux and macOS I should be able to get OpenSSL 3.0 working across the board.

On Windows, I've only got OpenSSL 3.0 working on CPython 3.11. It looks like the latest CPython releases only support OpenSSL 3.0 in the Visual Studio build files on 3.11+. I'm guessing upstream will be backporting OpenSSL 3.0 Windows support to older CPython since OpenSSL 1.1 is EOL and the community will likely throw a fit if CPython ships OpenSSL 1.1 with known security vulnerabilities (once an unpatched vuln in 1.1 inevitably drops). I'm content waiting for upstream to provide official OpenSSL 3.x support before shipping it in python-build-standalone.

If someone knows what CPython is actually planning for OpenSSL 3 on Windows, please comment!

[erlend-aasland]

If someone knows what CPython is actually planning for OpenSSL 3 on Windows, please comment!

Official CPython binaries from python.org are now using OpenSSL 3 for Windows builds (for the main, 3.12, and 3.11 branches).

[indygreg]

Official CPython binaries from python.org are now using OpenSSL 3 for Windows builds (for the main, 3.12, and 3.11 branches).

I realize this.

But CPython <3.11 are still in support. And OpenSSL 1.1 is out of support. So what's CPython's plans for OpenSSL 3.x on <3.11? Or put another way, what is CPython going to do when an OpenSSL 1.1 vulnerability drops? Are CPython <3.11 going to remain exposed? Is CPython going to distribute a patched OpenSSL 1.1?

[erlend-aasland]

Quoting python/cpython#99079 (comment):

None: binaries are no longer produced for these versions, and the code is already compatible (enough).

I think that answers most of your questions.

[Edward-Knight]

@indygreg I've pushed my WIP code to #189 - hopefully it will be useful to you. I've been working on it on-and-off in my fork for a while but haven't managed to get all the architectures working unfortunately