setup.py install doesn't work for packages with setup_requires on older macOS

[benjaminjkraft]

SInce the TLS 1.0/1.1 turndown, easy_install doesn't work on older macOS, because the builtin libraries don't support SSL. For the most part, we just use latest pip instead, and things work fine. But when installing a package with setup_requires from source (e.g. pip install git+git://github.com/pycqa/flake8.git@master), we run into trouble even on latest pip/setuptools, because pip runs setup.py install, which in turn runs easy_install, which fails. If easy_install is completely deprecated, setuptools should call out to pip instead; or easy_install do what pip does to make TLS 1.2 work.

See also pypa/packaging-problems#134.

[jaraco]

Once pip with PEP 518 support is released, setup_requires become deprecated. So the best recommendation is for packages to declare their build dependencies in their pyproject.toml and for users to use pip 10 and install their packages that way. The pyproject.toml should ensure the build requirements are present, preventing the need for setuptools to invoke easy_install.

Also, am I right in thinking this issue only happens to very old Pythons, than 2.7.9+ supports the requisite TLS? In that case, another workaround is simply not to use old Pythons (if you need 2.7, get it from Homebrew, but really you should be using Python 3).

And if you're really in a constrained environment where you're stuck on old Pythons, consider deploying your own devpi instance to provide compatibility.

If it were straightforward to support TLS 1.2 in easy_install or to somehow rely on pip to install setup_requires, I'd be in support of either, but my suspicion is that either of those approaches are fraught with bootstrapping challenges, so it's not worth the effort in light of the alternatives above.

[benjaminjkraft]

It's happening on 2.7.10 which is system python on Sierra -- I believe it's a matter of openssl versions, not python versions. Switching to homebrew's python does fix it; I think they must ship their own openssl. (We are stuck on python 2 at present.)

But anyway, totally fair, and thanks for the response. We'll keep an eye out for pip 10 in the meantime.

[ilovezfs]

@jaraco the suggestion in pypa/packaging-problems#134 (comment) is to fall back on SecureTransport as pip does.

[pudquick]

For explicit clarification: For any installs using the system-provided python on any macOS version earlier than 10.13.0, you will run into this issue not because of old python, but because those versions are linked against OpenSSL 0.9.8 and do not support TLS 1.1 or later.

easy_install would indeed need to be patched to swap in SecureTransport usage in a similar fashion.

[pganssle]

Are we going to fix this, or should we just say that setup_requires is deprecated now that pip has PEP 518 support?

[jaraco]

It turns out PIP 518/517 support isn't sufficient, due to usage of setup.py in other situations.

I ran into this issue on the latest macOS 10.14.2 with Python 3.5.4 from python.org (the latest available binary) using tox which invokes setup.py --name (tox-dev/tox#809). Probably this needs to be fixed.

[jaraco]

In this comment, Donald suggests:

use SecureTransport like pip has

Looking at the pip codebase, here's where SecureTransport is injected, but unfortunately, it relies on urllib3, and setuptools uses stdlib for its SSL, so Setuptools would need to be updated to use urllib3 to adopt this technique.

or an option that would fallback to pyOpenSSL

Hmm...