From 0818b058b65f8f863531c5cc9173c5b58c169953 Mon Sep 17 00:00:00 2001 From: Lance Goyke Date: Sat, 18 Nov 2023 07:38:18 -0600 Subject: [PATCH 1/8] compiling --- getting-started/setup-building.rst | 295 +++++++++++++++-------------- 1 file changed, 149 insertions(+), 146 deletions(-) diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst index e1a53a498..6e7ec87cf 100644 --- a/getting-started/setup-building.rst +++ b/getting-started/setup-building.rst @@ -143,199 +143,202 @@ checks that one should not skip. .. seealso:: The effects of various configure and build flags are documented in the `Python configure docs `_. -.. _unix-compiling: +.. tab:: Unix/macOS -Unix ----- + .. _unix-compiling: -The core CPython interpreter only needs a C compiler to be built, -however, some of the extension modules will need development headers -for additional libraries (such as the ``zlib`` library for compression). -Depending on what you intend to work on, you might need to install these -additional requirements so that the compiled interpreter supports the -desired features. + Unix + ---- -If you want to install these optional dependencies, consult the -:ref:`build-dependencies` section below. + The core CPython interpreter only needs a C compiler to be built, + however, some of the extension modules will need development headers + for additional libraries (such as the ``zlib`` library for compression). + Depending on what you intend to work on, you might need to install these + additional requirements so that the compiled interpreter supports the + desired features. -If you don't need to install them, the basic steps for building Python -for development is to configure it and then compile it. + If you want to install these optional dependencies, consult the + :ref:`build-dependencies` section below. -Configuration is typically:: + If you don't need to install them, the basic steps for building Python + for development is to configure it and then compile it. - $ ./configure --with-pydebug + Configuration is typically:: -More flags are available to ``configure``, but this is the minimum you should -do to get a pydebug build of CPython. + $ ./configure --with-pydebug -.. note:: - You might need to run ``make clean`` before or after re-running ``configure`` - in a particular build directory. + More flags are available to ``configure``, but this is the minimum you should + do to get a pydebug build of CPython. -Once ``configure`` is done, you can then compile CPython with:: + .. note:: + You might need to run ``make clean`` before or after re-running ``configure`` + in a particular build directory. - $ make -s -j2 + Once ``configure`` is done, you can then compile CPython with:: -This will build CPython with only warnings and errors being printed to -stderr and utilize up to 2 CPU cores. If you are using a multi-core machine -with more than 2 cores (or a single-core machine), you can adjust the number -passed into the ``-j`` flag to match the number of cores you have (or if your -version of Make supports it, you can use ``-j`` without a number and Make -will not limit the number of steps that can run simultaneously.). + $ make -s -j2 -At the end of the build you should see a success message, followed -by a list of extension modules that haven't been built because their -dependencies were missing: + This will build CPython with only warnings and errors being printed to + stderr and utilize up to 2 CPU cores. If you are using a multi-core machine + with more than 2 cores (or a single-core machine), you can adjust the number + passed into the ``-j`` flag to match the number of cores you have (or if your + version of Make supports it, you can use ``-j`` without a number and Make + will not limit the number of steps that can run simultaneously.). -.. code-block:: none + At the end of the build you should see a success message, followed + by a list of extension modules that haven't been built because their + dependencies were missing: - The necessary bits to build these optional modules were not found: - _gdbm - To find the necessary bits, look in configure.ac and config.log. + .. code-block:: none - Checked 106 modules (31 built-in, 74 shared, 0 n/a on macosx-13.4-arm64, 0 disabled, 1 missing, 0 failed on import) + The necessary bits to build these optional modules were not found: + _gdbm + To find the necessary bits, look in configure.ac and config.log. -If the build failed and you are using a C89 or C99-compliant compiler, -please open a bug report on the `issue tracker`_. + Checked 106 modules (31 built-in, 74 shared, 0 n/a on macosx-13.4-arm64, 0 disabled, 1 missing, 0 failed on import) -If you decide to :ref:`build-dependencies`, you will need to re-run both -``configure`` and ``make``. + If the build failed and you are using a C89 or C99-compliant compiler, + please open a bug report on the `issue tracker`_. -.. _mac-python.exe: + If you decide to :ref:`build-dependencies`, you will need to re-run both + ``configure`` and ``make``. -Once CPython is done building you will then have a working build -that can be run in-place; ``./python`` on most machines (and what is used in -all examples), ``./python.exe`` wherever a case-insensitive filesystem is used -(e.g. on macOS by default), in order to avoid conflicts with the ``Python`` -directory. There is normally no need to install your built copy -of Python! The interpreter will realize where it is being run from -and thus use the files found in the working copy. If you are worried -you might accidentally install your working copy build, you can add -``--prefix=/tmp/python`` to the configuration step. When running from your -working directory, it is best to avoid using the ``--enable-shared`` flag -to ``configure``; unless you are very careful, you may accidentally run -with code from an older, installed shared Python library rather than from -the interpreter you just built. + .. _mac-python.exe: + Once CPython is done building you will then have a working build + that can be run in-place; ``./python`` on most machines (and what is used in + all examples), ``./python.exe`` wherever a case-insensitive filesystem is used + (e.g. on macOS by default), in order to avoid conflicts with the ``Python`` + directory. There is normally no need to install your built copy + of Python! The interpreter will realize where it is being run from + and thus use the files found in the working copy. If you are worried + you might accidentally install your working copy build, you can add + ``--prefix=/tmp/python`` to the configuration step. When running from your + working directory, it is best to avoid using the ``--enable-shared`` flag + to ``configure``; unless you are very careful, you may accidentally run + with code from an older, installed shared Python library rather than from + the interpreter you just built. -Clang -''''' -If you are using clang_ to build CPython, some flags you might want to set to -quiet some standard warnings which are specifically superfluous to CPython are -``-Wno-unused-value -Wno-empty-body -Qunused-arguments``. You can set your -``CFLAGS`` environment variable to these flags when running ``configure``. + Clang + ''''' -If you are using clang_ with ccache_, turn off the noisy -``parentheses-equality`` warnings with the ``-Wno-parentheses-equality`` flag. -These warnings are caused by clang not having enough information to detect -that extraneous parentheses in expanded macros are valid, because the -preprocessing is done separately by ccache. + If you are using clang_ to build CPython, some flags you might want to set to + quiet some standard warnings which are specifically superfluous to CPython are + ``-Wno-unused-value -Wno-empty-body -Qunused-arguments``. You can set your + ``CFLAGS`` environment variable to these flags when running ``configure``. -If you are using LLVM 2.8, also use the ``-no-integrated-as`` flag in order to -build the :py:mod:`ctypes` module (without the flag the rest of CPython will -still build properly). + If you are using clang_ with ccache_, turn off the noisy + ``parentheses-equality`` warnings with the ``-Wno-parentheses-equality`` flag. + These warnings are caused by clang not having enough information to detect + that extraneous parentheses in expanded macros are valid, because the + preprocessing is done separately by ccache. + If you are using LLVM 2.8, also use the ``-no-integrated-as`` flag in order to + build the :py:mod:`ctypes` module (without the flag the rest of CPython will + still build properly). -Optimization -'''''''''''' -If you are trying to improve CPython's performance, you will probably want -to use an optimized build of CPython. It can take a lot longer to build CPython -with optimizations enabled, and it's usually not necessary to do so. However, -it's essential if you want accurate benchmark results for a proposed performance -optimization. + Optimization + '''''''''''' -For an optimized build of Python, use -``configure --enable-optimizations --with-lto``. -This sets the default make targets up to enable Profile Guided Optimization (PGO) -and may be used to auto-enable Link Time Optimization (LTO) on some platforms. -See :option:`python:--enable-optimizations` and :option:`python:--with-lto` -to learn more about these options. + If you are trying to improve CPython's performance, you will probably want + to use an optimized build of CPython. It can take a lot longer to build CPython + with optimizations enabled, and it's usually not necessary to do so. However, + it's essential if you want accurate benchmark results for a proposed performance + optimization. -.. code:: console + For an optimized build of Python, use + ``configure --enable-optimizations --with-lto``. + This sets the default make targets up to enable Profile Guided Optimization (PGO) + and may be used to auto-enable Link Time Optimization (LTO) on some platforms. + See :option:`python:--enable-optimizations` and :option:`python:--with-lto` + to learn more about these options. - $ ./configure --enable-optimizations --with-lto + .. code:: console + $ ./configure --enable-optimizations --with-lto -.. _windows-compiling: +.. tab:: Windows -Windows -------- + .. _windows-compiling: -.. note:: If you are using the Windows Subsystem for Linux (WSL), - :ref:`clone the repository ` from a native Windows shell program - like PowerShell or the ``cmd.exe`` command prompt, - and use a build of Git targeted for Windows, - e.g. the `Git for Windows download from the official Git website`_. - Otherwise, Visual Studio will not be able to find all the project's files - and will fail the build. + Windows + ------- -For a concise step by step summary of building Python on Windows, -you can read `Victor Stinner's guide`_. + .. note:: If you are using the Windows Subsystem for Linux (WSL), + :ref:`clone the repository ` from a native Windows shell program + like PowerShell or the ``cmd.exe`` command prompt, + and use a build of Git targeted for Windows, + e.g. the `Git for Windows download from the official Git website`_. + Otherwise, Visual Studio will not be able to find all the project's files + and will fail the build. -All supported versions of Python can be built -using Microsoft Visual Studio 2017 or later. -You can download and use any of the free or paid versions of `Visual Studio`_. - -When installing it, select the :guilabel:`Python development` workload -and the optional :guilabel:`Python native development tools` component -to obtain all of the necessary build tools. -You can find Git for Windows on the :guilabel:`Individual components` tab -if you don't already have it installed. + For a concise step by step summary of building Python on Windows, + you can read `Victor Stinner's guide`_. -.. note:: If you want to build MSI installers, be aware that the build toolchain - for them has a dependency on the Microsoft .NET Framework Version 3.5 - (which may not be included on recent versions of Windows, such as Windows 10). - If you are building on a recent Windows version, use the Control Panel - (:menuselection:`Programs --> Programs and Features --> Turn Windows Features on or off`) - and ensure that the entry - :guilabel:`.NET Framework 3.5 (includes .NET 2.0 and 3.0)` is enabled. + All supported versions of Python can be built + using Microsoft Visual Studio 2017 or later. + You can download and use any of the free or paid versions of `Visual Studio`_. -Your first build should use the command line to ensure any external dependencies -are downloaded: + When installing it, select the :guilabel:`Python development` workload + and the optional :guilabel:`Python native development tools` component + to obtain all of the necessary build tools. + You can find Git for Windows on the :guilabel:`Individual components` tab + if you don't already have it installed. -.. code-block:: batch - - PCbuild\build.bat -c Debug - -The above command line build uses the ``-c Debug`` argument -to build in the ``Debug`` configuration, -which enables checks and assertions helpful for developing Python. -By default, it builds in the ``Release`` configuration -and for the 64-bit ``x64`` platform rather than 32-bit ``Win32``; -use ``-c`` and ``-p`` to control build config and platform, respectively. - -After this build succeeds, you can open the ``PCbuild\pcbuild.sln`` solution -in the Visual Studio IDE to continue development, if you prefer. -When building in Visual Studio, -make sure to select build settings that match what you used with the script -(the :guilabel:`Debug` configuration and the :guilabel:`x64` platform) -from the dropdown menus in the toolbar. + .. note:: If you want to build MSI installers, be aware that the build toolchain + for them has a dependency on the Microsoft .NET Framework Version 3.5 + (which may not be included on recent versions of Windows, such as Windows 10). + If you are building on a recent Windows version, use the Control Panel + (:menuselection:`Programs --> Programs and Features --> Turn Windows Features on or off`) + and ensure that the entry + :guilabel:`.NET Framework 3.5 (includes .NET 2.0 and 3.0)` is enabled. -.. note:: + Your first build should use the command line to ensure any external dependencies + are downloaded: + + .. code-block:: batch + + PCbuild\build.bat -c Debug + + The above command line build uses the ``-c Debug`` argument + to build in the ``Debug`` configuration, + which enables checks and assertions helpful for developing Python. + By default, it builds in the ``Release`` configuration + and for the 64-bit ``x64`` platform rather than 32-bit ``Win32``; + use ``-c`` and ``-p`` to control build config and platform, respectively. + + After this build succeeds, you can open the ``PCbuild\pcbuild.sln`` solution + in the Visual Studio IDE to continue development, if you prefer. + When building in Visual Studio, + make sure to select build settings that match what you used with the script + (the :guilabel:`Debug` configuration and the :guilabel:`x64` platform) + from the dropdown menus in the toolbar. + + .. note:: - If you need to change the build configuration or platform, - build once with the ``build.bat`` script set to those options first - before building with them in VS to ensure all files are rebuilt properly, - or you may encounter errors when loading modules that were not rebuilt. + If you need to change the build configuration or platform, + build once with the ``build.bat`` script set to those options first + before building with them in VS to ensure all files are rebuilt properly, + or you may encounter errors when loading modules that were not rebuilt. - Avoid selecting the ``PGInstrument`` and ``PGUpdate`` configurations, - as these are intended for PGO builds and not for normal development. + Avoid selecting the ``PGInstrument`` and ``PGUpdate`` configurations, + as these are intended for PGO builds and not for normal development. -You can run the build of Python you've compiled with: + You can run the build of Python you've compiled with: -.. code-block:: batch + .. code-block:: batch - PCbuild\amd64\python_d.exe + PCbuild\amd64\python_d.exe -See the `PCBuild readme`_ for more details on what other software is necessary -and how to build. + See the `PCBuild readme`_ for more details on what other software is necessary + and how to build. -.. _Victor Stinner's guide: https://web.archive.org/web/20220907075854/https://cpython-core-tutorial.readthedocs.io/en/latest/build_cpython_windows.html -.. _Visual Studio: https://visualstudio.microsoft.com/ -.. _PCBuild readme: https://github.com/python/cpython/blob/main/PCbuild/readme.txt -.. _Git for Windows download from the official Git website: https://git-scm.com/download/win + .. _Victor Stinner's guide: https://web.archive.org/web/20220907075854/https://cpython-core-tutorial.readthedocs.io/en/latest/build_cpython_windows.html + .. _Visual Studio: https://visualstudio.microsoft.com/ + .. _PCBuild readme: https://github.com/python/cpython/blob/main/PCbuild/readme.txt + .. _Git for Windows download from the official Git website: https://git-scm.com/download/win .. _build-dependencies: From f7c5cedd58ca739ce35ce235811d12409e8c76be Mon Sep 17 00:00:00 2001 From: Lance Goyke Date: Sat, 18 Nov 2023 07:46:59 -0600 Subject: [PATCH 2/8] install dependencies --- getting-started/setup-building.rst | 194 +++++++++++++++-------------- 1 file changed, 99 insertions(+), 95 deletions(-) diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst index 6e7ec87cf..f997e4fdc 100644 --- a/getting-started/setup-building.rst +++ b/getting-started/setup-building.rst @@ -350,148 +350,152 @@ This section explains how to install additional extensions (e.g. ``zlib``) on :ref:`Linux ` and :ref:`macOS`. On Windows, extensions are already included and built automatically. -.. _deps-on-linux: +.. tab:: Linux -Linux ------ + .. _deps-on-linux: -For Unix-based systems, we try to use system libraries whenever available. -This means optional components will only build if the relevant system headers -are available. The best way to obtain the appropriate headers will vary by -distribution, but the appropriate commands for some popular distributions -are below. + Linux + ----- -On **Fedora**, **Red Hat Enterprise Linux** and other ``yum`` based systems:: + For Unix-based systems, we try to use system libraries whenever available. + This means optional components will only build if the relevant system headers + are available. The best way to obtain the appropriate headers will vary by + distribution, but the appropriate commands for some popular distributions + are below. - $ sudo yum install yum-utils - $ sudo yum-builddep python3 + On **Fedora**, **Red Hat Enterprise Linux** and other ``yum`` based systems:: -On **Fedora** and other ``DNF`` based systems:: + $ sudo yum install yum-utils + $ sudo yum-builddep python3 - $ sudo dnf install dnf-plugins-core # install this to use 'dnf builddep' - $ sudo dnf builddep python3 + On **Fedora** and other ``DNF`` based systems:: -On **Debian**, **Ubuntu**, and other ``apt`` based systems, try to get the -dependencies for the Python you're working on by using the ``apt`` command. + $ sudo dnf install dnf-plugins-core # install this to use 'dnf builddep' + $ sudo dnf builddep python3 -First, make sure you have enabled the source packages in the sources list. -You can do this by adding the location of the source packages, including -URL, distribution name and component name, to ``/etc/apt/sources.list``. -Take Ubuntu 22.04 LTS (Jammy Jellyfish) for example:: + On **Debian**, **Ubuntu**, and other ``apt`` based systems, try to get the + dependencies for the Python you're working on by using the ``apt`` command. - deb-src http://archive.ubuntu.com/ubuntu/ jammy main + First, make sure you have enabled the source packages in the sources list. + You can do this by adding the location of the source packages, including + URL, distribution name and component name, to ``/etc/apt/sources.list``. + Take Ubuntu 22.04 LTS (Jammy Jellyfish) for example:: -Alternatively, uncomment lines with ``deb-src`` using an editor, e.g.:: + deb-src http://archive.ubuntu.com/ubuntu/ jammy main - sudo nano /etc/apt/sources.list + Alternatively, uncomment lines with ``deb-src`` using an editor, e.g.:: -For other distributions, like Debian, change the URL and names to correspond -with the specific distribution. + sudo nano /etc/apt/sources.list -Then you should update the packages index:: + For other distributions, like Debian, change the URL and names to correspond + with the specific distribution. - $ sudo apt-get update + Then you should update the packages index:: -Now you can install the build dependencies via ``apt``:: + $ sudo apt-get update - $ sudo apt-get build-dep python3 - $ sudo apt-get install pkg-config + Now you can install the build dependencies via ``apt``:: -If you want to build all optional modules, install the following packages and -their dependencies:: + $ sudo apt-get build-dep python3 + $ sudo apt-get install pkg-config - $ sudo apt-get install build-essential gdb lcov pkg-config \ - libbz2-dev libffi-dev libgdbm-dev libgdbm-compat-dev liblzma-dev \ - libncurses5-dev libreadline6-dev libsqlite3-dev libssl-dev \ - lzma lzma-dev tk-dev uuid-dev zlib1g-dev + If you want to build all optional modules, install the following packages and + their dependencies:: + $ sudo apt-get install build-essential gdb lcov pkg-config \ + libbz2-dev libffi-dev libgdbm-dev libgdbm-compat-dev liblzma-dev \ + libncurses5-dev libreadline6-dev libsqlite3-dev libssl-dev \ + lzma lzma-dev tk-dev uuid-dev zlib1g-dev -.. _macOS and OS X: -.. _macOS: -macOS ------ +.. tab:: macOS -For **macOS systems** (versions 10.9+), -the Developer Tools can be downloaded and installed automatically; -you do not need to download the complete Xcode application. + .. _macOS and OS X: + .. _macOS: -If necessary, run the following:: + macOS + ----- - $ xcode-select --install + For **macOS systems** (versions 10.9+), + the Developer Tools can be downloaded and installed automatically; + you do not need to download the complete Xcode application. -This will also ensure that the system header files are installed into -``/usr/include``. + If necessary, run the following:: -.. _Homebrew: https://brew.sh + $ xcode-select --install -.. _MacPorts: https://www.macports.org + This will also ensure that the system header files are installed into + ``/usr/include``. -Also note that macOS does not include several libraries used by the Python -standard library, including ``libzma``, so expect to see some extension module -build failures unless you install local copies of them. As of OS X 10.11, -Apple no longer provides header files for the deprecated system version of -OpenSSL which means that you will not be able to build the ``_ssl`` extension. -One solution is to install these libraries from a third-party package -manager, like Homebrew_ or MacPorts_, and then add the appropriate paths -for the header and library files to your ``configure`` command. + .. _Homebrew: https://brew.sh -For example, with **Homebrew**, install the dependencies:: + .. _MacPorts: https://www.macports.org - $ brew install pkg-config openssl@3.0 xz gdbm tcl-tk + Also note that macOS does not include several libraries used by the Python + standard library, including ``libzma``, so expect to see some extension module + build failures unless you install local copies of them. As of OS X 10.11, + Apple no longer provides header files for the deprecated system version of + OpenSSL which means that you will not be able to build the ``_ssl`` extension. + One solution is to install these libraries from a third-party package + manager, like Homebrew_ or MacPorts_, and then add the appropriate paths + for the header and library files to your ``configure`` command. -Then, for Python 3.11 and newer, run ``configure``:: + For example, with **Homebrew**, install the dependencies:: - $ GDBM_CFLAGS="-I$(brew --prefix gdbm)/include" \ - GDBM_LIBS="-L$(brew --prefix gdbm)/lib -lgdbm" \ - ./configure --with-pydebug \ - --with-openssl="$(brew --prefix openssl@3.0)" + $ brew install pkg-config openssl@3.0 xz gdbm tcl-tk -Or, for Python 3.8 through 3.10:: + Then, for Python 3.11 and newer, run ``configure``:: - $ CPPFLAGS="-I$(brew --prefix gdbm)/include -I$(brew --prefix xz)/include" \ - LDFLAGS="-L$(brew --prefix gdbm)/lib -L$(brew --prefix xz)/lib" \ - ./configure --with-pydebug \ - --with-openssl="$(brew --prefix openssl@3.0)" \ - --with-tcltk-libs="$(pkg-config --libs tcl tk)" \ - --with-tcltk-includes="$(pkg-config --cflags tcl tk)" + $ GDBM_CFLAGS="-I$(brew --prefix gdbm)/include" \ + GDBM_LIBS="-L$(brew --prefix gdbm)/lib -lgdbm" \ + ./configure --with-pydebug \ + --with-openssl="$(brew --prefix openssl@3.0)" -And finally, run ``make``:: + Or, for Python 3.8 through 3.10:: - $ make -s -j2 + $ CPPFLAGS="-I$(brew --prefix gdbm)/include -I$(brew --prefix xz)/include" \ + LDFLAGS="-L$(brew --prefix gdbm)/lib -L$(brew --prefix xz)/lib" \ + ./configure --with-pydebug \ + --with-openssl="$(brew --prefix openssl@3.0)" \ + --with-tcltk-libs="$(pkg-config --libs tcl tk)" \ + --with-tcltk-includes="$(pkg-config --cflags tcl tk)" -Alternatively, with **MacPorts**:: + And finally, run ``make``:: - $ sudo port install pkgconfig openssl xz gdbm tcl tk +quartz + $ make -s -j2 + + Alternatively, with **MacPorts**:: -Then, for Python 3.11 and newer, run ``configure``:: + $ sudo port install pkgconfig openssl xz gdbm tcl tk +quartz - $ GDBM_CFLAGS="-I$(dirname $(dirname $(which port)))/include" \ - GDBM_LIBS="-L$(dirname $(dirname $(which port)))/lib -lgdbm" \ - ./configure --with-pydebug + Then, for Python 3.11 and newer, run ``configure``:: -And finally, run ``make``:: + $ GDBM_CFLAGS="-I$(dirname $(dirname $(which port)))/include" \ + GDBM_LIBS="-L$(dirname $(dirname $(which port)))/lib -lgdbm" \ + ./configure --with-pydebug - $ make -s -j2 + And finally, run ``make``:: + + $ make -s -j2 -There will sometimes be optional modules added for a new release which -won't yet be identified in the OS-level build dependencies. In those cases, -just ask for assistance in the *Core Development* category on :ref:`help-discourse`. + There will sometimes be optional modules added for a new release which + won't yet be identified in the OS-level build dependencies. In those cases, + just ask for assistance in the *Core Development* category on :ref:`help-discourse`. -Explaining how to build optional dependencies on a Unix-based system without -root access is beyond the scope of this guide. + Explaining how to build optional dependencies on a Unix-based system without + root access is beyond the scope of this guide. -For more details on various options and considerations for building, refer -to the `macOS README -`_. + For more details on various options and considerations for building, refer + to the `macOS README + `_. -.. _clang: https://clang.llvm.org/ -.. _ccache: https://ccache.dev/ + .. _clang: https://clang.llvm.org/ + .. _ccache: https://ccache.dev/ -.. note:: While you need a C compiler to build CPython, you don't need any - knowledge of the C language to contribute! Vast areas of CPython are - written completely in Python: as of this writing, CPython contains slightly - more Python code than C. + .. note:: While you need a C compiler to build CPython, you don't need any + knowledge of the C language to contribute! Vast areas of CPython are + written completely in Python: as of this writing, CPython contains slightly + more Python code than C. .. _regenerate_configure: From 5ab7b727e668686ef48318375be76ac08a32383c Mon Sep 17 00:00:00 2001 From: Lance Goyke Date: Sat, 18 Nov 2023 07:51:10 -0600 Subject: [PATCH 3/8] add short tab about windows --- getting-started/setup-building.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst index f997e4fdc..230347d4a 100644 --- a/getting-started/setup-building.rst +++ b/getting-started/setup-building.rst @@ -497,6 +497,10 @@ extensions are already included and built automatically. written completely in Python: as of this writing, CPython contains slightly more Python code than C. +.. tab:: Windows + + On Windows, extensions are already included and built automatically. + .. _regenerate_configure: From 1400c2aa4ea1f1901528f1b4b64788f1efa26c33 Mon Sep 17 00:00:00 2001 From: Lance Goyke Date: Sat, 18 Nov 2023 07:52:42 -0600 Subject: [PATCH 4/8] change Linux tab to Unix --- getting-started/setup-building.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst index 230347d4a..4886059d8 100644 --- a/getting-started/setup-building.rst +++ b/getting-started/setup-building.rst @@ -350,7 +350,7 @@ This section explains how to install additional extensions (e.g. ``zlib``) on :ref:`Linux ` and :ref:`macOS`. On Windows, extensions are already included and built automatically. -.. tab:: Linux +.. tab:: Unix .. _deps-on-linux: From 6d158c7eb23b4deb49bf30d9178fa5b278dc95b3 Mon Sep 17 00:00:00 2001 From: Lance Goyke Date: Mon, 20 Nov 2023 08:20:31 -0600 Subject: [PATCH 5/8] move references out of Install depencies tabs --- getting-started/setup-building.rst | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-) diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst index 4886059d8..b8b1b5610 100644 --- a/getting-started/setup-building.rst +++ b/getting-started/setup-building.rst @@ -342,6 +342,9 @@ checks that one should not skip. .. _build-dependencies: +.. _deps-on-linux: +.. _macOS and OS X: +.. _macOS: Install dependencies ==================== @@ -350,12 +353,7 @@ This section explains how to install additional extensions (e.g. ``zlib``) on :ref:`Linux ` and :ref:`macOS`. On Windows, extensions are already included and built automatically. -.. tab:: Unix - - .. _deps-on-linux: - - Linux - ----- +.. tab:: Linux For Unix-based systems, we try to use system libraries whenever available. This means optional components will only build if the relevant system headers @@ -410,12 +408,6 @@ extensions are already included and built automatically. .. tab:: macOS - .. _macOS and OS X: - .. _macOS: - - macOS - ----- - For **macOS systems** (versions 10.9+), the Developer Tools can be downloaded and installed automatically; you do not need to download the complete Xcode application. From 81d1dc13775a8cf77d5b0b1c7045d6f284bfc6e5 Mon Sep 17 00:00:00 2001 From: Lance Goyke Date: Mon, 20 Nov 2023 08:22:05 -0600 Subject: [PATCH 6/8] remove tabs from Compile and build section --- getting-started/setup-building.rst | 306 ++++++++++++++--------------- 1 file changed, 151 insertions(+), 155 deletions(-) diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst index b8b1b5610..4f19aabdf 100644 --- a/getting-started/setup-building.rst +++ b/getting-started/setup-building.rst @@ -143,202 +143,198 @@ checks that one should not skip. .. seealso:: The effects of various configure and build flags are documented in the `Python configure docs `_. -.. tab:: Unix/macOS +.. _unix-compiling: - .. _unix-compiling: +Unix +---- - Unix - ---- +The core CPython interpreter only needs a C compiler to be built, +however, some of the extension modules will need development headers +for additional libraries (such as the ``zlib`` library for compression). +Depending on what you intend to work on, you might need to install these +additional requirements so that the compiled interpreter supports the +desired features. - The core CPython interpreter only needs a C compiler to be built, - however, some of the extension modules will need development headers - for additional libraries (such as the ``zlib`` library for compression). - Depending on what you intend to work on, you might need to install these - additional requirements so that the compiled interpreter supports the - desired features. +If you want to install these optional dependencies, consult the +:ref:`build-dependencies` section below. - If you want to install these optional dependencies, consult the - :ref:`build-dependencies` section below. +If you don't need to install them, the basic steps for building Python +for development is to configure it and then compile it. - If you don't need to install them, the basic steps for building Python - for development is to configure it and then compile it. +Configuration is typically:: - Configuration is typically:: + $ ./configure --with-pydebug - $ ./configure --with-pydebug +More flags are available to ``configure``, but this is the minimum you should +do to get a pydebug build of CPython. - More flags are available to ``configure``, but this is the minimum you should - do to get a pydebug build of CPython. - - .. note:: - You might need to run ``make clean`` before or after re-running ``configure`` - in a particular build directory. - - Once ``configure`` is done, you can then compile CPython with:: - - $ make -s -j2 - - This will build CPython with only warnings and errors being printed to - stderr and utilize up to 2 CPU cores. If you are using a multi-core machine - with more than 2 cores (or a single-core machine), you can adjust the number - passed into the ``-j`` flag to match the number of cores you have (or if your - version of Make supports it, you can use ``-j`` without a number and Make - will not limit the number of steps that can run simultaneously.). - - At the end of the build you should see a success message, followed - by a list of extension modules that haven't been built because their - dependencies were missing: - - .. code-block:: none - - The necessary bits to build these optional modules were not found: - _gdbm - To find the necessary bits, look in configure.ac and config.log. - - Checked 106 modules (31 built-in, 74 shared, 0 n/a on macosx-13.4-arm64, 0 disabled, 1 missing, 0 failed on import) - - If the build failed and you are using a C89 or C99-compliant compiler, - please open a bug report on the `issue tracker`_. - - If you decide to :ref:`build-dependencies`, you will need to re-run both - ``configure`` and ``make``. - - .. _mac-python.exe: +.. note:: + You might need to run ``make clean`` before or after re-running ``configure`` + in a particular build directory. - Once CPython is done building you will then have a working build - that can be run in-place; ``./python`` on most machines (and what is used in - all examples), ``./python.exe`` wherever a case-insensitive filesystem is used - (e.g. on macOS by default), in order to avoid conflicts with the ``Python`` - directory. There is normally no need to install your built copy - of Python! The interpreter will realize where it is being run from - and thus use the files found in the working copy. If you are worried - you might accidentally install your working copy build, you can add - ``--prefix=/tmp/python`` to the configuration step. When running from your - working directory, it is best to avoid using the ``--enable-shared`` flag - to ``configure``; unless you are very careful, you may accidentally run - with code from an older, installed shared Python library rather than from - the interpreter you just built. +Once ``configure`` is done, you can then compile CPython with:: + $ make -s -j2 - Clang - ''''' +This will build CPython with only warnings and errors being printed to +stderr and utilize up to 2 CPU cores. If you are using a multi-core machine +with more than 2 cores (or a single-core machine), you can adjust the number +passed into the ``-j`` flag to match the number of cores you have (or if your +version of Make supports it, you can use ``-j`` without a number and Make +will not limit the number of steps that can run simultaneously.). - If you are using clang_ to build CPython, some flags you might want to set to - quiet some standard warnings which are specifically superfluous to CPython are - ``-Wno-unused-value -Wno-empty-body -Qunused-arguments``. You can set your - ``CFLAGS`` environment variable to these flags when running ``configure``. +At the end of the build you should see a success message, followed +by a list of extension modules that haven't been built because their +dependencies were missing: - If you are using clang_ with ccache_, turn off the noisy - ``parentheses-equality`` warnings with the ``-Wno-parentheses-equality`` flag. - These warnings are caused by clang not having enough information to detect - that extraneous parentheses in expanded macros are valid, because the - preprocessing is done separately by ccache. +.. code-block:: none + + The necessary bits to build these optional modules were not found: + _gdbm + To find the necessary bits, look in configure.ac and config.log. + + Checked 106 modules (31 built-in, 74 shared, 0 n/a on macosx-13.4-arm64, 0 disabled, 1 missing, 0 failed on import) - If you are using LLVM 2.8, also use the ``-no-integrated-as`` flag in order to - build the :py:mod:`ctypes` module (without the flag the rest of CPython will - still build properly). +If the build failed and you are using a C89 or C99-compliant compiler, +please open a bug report on the `issue tracker`_. +If you decide to :ref:`build-dependencies`, you will need to re-run both +``configure`` and ``make``. - Optimization - '''''''''''' +.. _mac-python.exe: - If you are trying to improve CPython's performance, you will probably want - to use an optimized build of CPython. It can take a lot longer to build CPython - with optimizations enabled, and it's usually not necessary to do so. However, - it's essential if you want accurate benchmark results for a proposed performance - optimization. +Once CPython is done building you will then have a working build +that can be run in-place; ``./python`` on most machines (and what is used in +all examples), ``./python.exe`` wherever a case-insensitive filesystem is used +(e.g. on macOS by default), in order to avoid conflicts with the ``Python`` +directory. There is normally no need to install your built copy +of Python! The interpreter will realize where it is being run from +and thus use the files found in the working copy. If you are worried +you might accidentally install your working copy build, you can add +``--prefix=/tmp/python`` to the configuration step. When running from your +working directory, it is best to avoid using the ``--enable-shared`` flag +to ``configure``; unless you are very careful, you may accidentally run +with code from an older, installed shared Python library rather than from +the interpreter you just built. - For an optimized build of Python, use - ``configure --enable-optimizations --with-lto``. - This sets the default make targets up to enable Profile Guided Optimization (PGO) - and may be used to auto-enable Link Time Optimization (LTO) on some platforms. - See :option:`python:--enable-optimizations` and :option:`python:--with-lto` - to learn more about these options. - .. code:: console +Clang +''''' - $ ./configure --enable-optimizations --with-lto +If you are using clang_ to build CPython, some flags you might want to set to +quiet some standard warnings which are specifically superfluous to CPython are +``-Wno-unused-value -Wno-empty-body -Qunused-arguments``. You can set your +``CFLAGS`` environment variable to these flags when running ``configure``. -.. tab:: Windows +If you are using clang_ with ccache_, turn off the noisy +``parentheses-equality`` warnings with the ``-Wno-parentheses-equality`` flag. +These warnings are caused by clang not having enough information to detect +that extraneous parentheses in expanded macros are valid, because the +preprocessing is done separately by ccache. - .. _windows-compiling: +If you are using LLVM 2.8, also use the ``-no-integrated-as`` flag in order to +build the :py:mod:`ctypes` module (without the flag the rest of CPython will +still build properly). - Windows - ------- - .. note:: If you are using the Windows Subsystem for Linux (WSL), - :ref:`clone the repository ` from a native Windows shell program - like PowerShell or the ``cmd.exe`` command prompt, - and use a build of Git targeted for Windows, - e.g. the `Git for Windows download from the official Git website`_. - Otherwise, Visual Studio will not be able to find all the project's files - and will fail the build. +Optimization +'''''''''''' - For a concise step by step summary of building Python on Windows, - you can read `Victor Stinner's guide`_. +If you are trying to improve CPython's performance, you will probably want +to use an optimized build of CPython. It can take a lot longer to build CPython +with optimizations enabled, and it's usually not necessary to do so. However, +it's essential if you want accurate benchmark results for a proposed performance +optimization. - All supported versions of Python can be built - using Microsoft Visual Studio 2017 or later. - You can download and use any of the free or paid versions of `Visual Studio`_. +For an optimized build of Python, use +``configure --enable-optimizations --with-lto``. +This sets the default make targets up to enable Profile Guided Optimization (PGO) +and may be used to auto-enable Link Time Optimization (LTO) on some platforms. +See :option:`python:--enable-optimizations` and :option:`python:--with-lto` +to learn more about these options. - When installing it, select the :guilabel:`Python development` workload - and the optional :guilabel:`Python native development tools` component - to obtain all of the necessary build tools. - You can find Git for Windows on the :guilabel:`Individual components` tab - if you don't already have it installed. +.. code:: console - .. note:: If you want to build MSI installers, be aware that the build toolchain - for them has a dependency on the Microsoft .NET Framework Version 3.5 - (which may not be included on recent versions of Windows, such as Windows 10). - If you are building on a recent Windows version, use the Control Panel - (:menuselection:`Programs --> Programs and Features --> Turn Windows Features on or off`) - and ensure that the entry - :guilabel:`.NET Framework 3.5 (includes .NET 2.0 and 3.0)` is enabled. + $ ./configure --enable-optimizations --with-lto - Your first build should use the command line to ensure any external dependencies - are downloaded: +.. _windows-compiling: - .. code-block:: batch +Windows +------- - PCbuild\build.bat -c Debug +.. note:: If you are using the Windows Subsystem for Linux (WSL), + :ref:`clone the repository ` from a native Windows shell program + like PowerShell or the ``cmd.exe`` command prompt, + and use a build of Git targeted for Windows, + e.g. the `Git for Windows download from the official Git website`_. + Otherwise, Visual Studio will not be able to find all the project's files + and will fail the build. - The above command line build uses the ``-c Debug`` argument - to build in the ``Debug`` configuration, - which enables checks and assertions helpful for developing Python. - By default, it builds in the ``Release`` configuration - and for the 64-bit ``x64`` platform rather than 32-bit ``Win32``; - use ``-c`` and ``-p`` to control build config and platform, respectively. +For a concise step by step summary of building Python on Windows, +you can read `Victor Stinner's guide`_. + +All supported versions of Python can be built +using Microsoft Visual Studio 2017 or later. +You can download and use any of the free or paid versions of `Visual Studio`_. + +When installing it, select the :guilabel:`Python development` workload +and the optional :guilabel:`Python native development tools` component +to obtain all of the necessary build tools. +You can find Git for Windows on the :guilabel:`Individual components` tab +if you don't already have it installed. - After this build succeeds, you can open the ``PCbuild\pcbuild.sln`` solution - in the Visual Studio IDE to continue development, if you prefer. - When building in Visual Studio, - make sure to select build settings that match what you used with the script - (the :guilabel:`Debug` configuration and the :guilabel:`x64` platform) - from the dropdown menus in the toolbar. +.. note:: If you want to build MSI installers, be aware that the build toolchain + for them has a dependency on the Microsoft .NET Framework Version 3.5 + (which may not be included on recent versions of Windows, such as Windows 10). + If you are building on a recent Windows version, use the Control Panel + (:menuselection:`Programs --> Programs and Features --> Turn Windows Features on or off`) + and ensure that the entry + :guilabel:`.NET Framework 3.5 (includes .NET 2.0 and 3.0)` is enabled. + +Your first build should use the command line to ensure any external dependencies +are downloaded: + +.. code-block:: batch + + PCbuild\build.bat -c Debug + +The above command line build uses the ``-c Debug`` argument +to build in the ``Debug`` configuration, +which enables checks and assertions helpful for developing Python. +By default, it builds in the ``Release`` configuration +and for the 64-bit ``x64`` platform rather than 32-bit ``Win32``; +use ``-c`` and ``-p`` to control build config and platform, respectively. + +After this build succeeds, you can open the ``PCbuild\pcbuild.sln`` solution +in the Visual Studio IDE to continue development, if you prefer. +When building in Visual Studio, +make sure to select build settings that match what you used with the script +(the :guilabel:`Debug` configuration and the :guilabel:`x64` platform) +from the dropdown menus in the toolbar. - .. note:: +.. note:: - If you need to change the build configuration or platform, - build once with the ``build.bat`` script set to those options first - before building with them in VS to ensure all files are rebuilt properly, - or you may encounter errors when loading modules that were not rebuilt. + If you need to change the build configuration or platform, + build once with the ``build.bat`` script set to those options first + before building with them in VS to ensure all files are rebuilt properly, + or you may encounter errors when loading modules that were not rebuilt. - Avoid selecting the ``PGInstrument`` and ``PGUpdate`` configurations, - as these are intended for PGO builds and not for normal development. + Avoid selecting the ``PGInstrument`` and ``PGUpdate`` configurations, + as these are intended for PGO builds and not for normal development. - You can run the build of Python you've compiled with: +You can run the build of Python you've compiled with: - .. code-block:: batch +.. code-block:: batch - PCbuild\amd64\python_d.exe + PCbuild\amd64\python_d.exe - See the `PCBuild readme`_ for more details on what other software is necessary - and how to build. +See the `PCBuild readme`_ for more details on what other software is necessary +and how to build. - .. _Victor Stinner's guide: https://web.archive.org/web/20220907075854/https://cpython-core-tutorial.readthedocs.io/en/latest/build_cpython_windows.html - .. _Visual Studio: https://visualstudio.microsoft.com/ - .. _PCBuild readme: https://github.com/python/cpython/blob/main/PCbuild/readme.txt - .. _Git for Windows download from the official Git website: https://git-scm.com/download/win +.. _Victor Stinner's guide: https://web.archive.org/web/20220907075854/https://cpython-core-tutorial.readthedocs.io/en/latest/build_cpython_windows.html +.. _Visual Studio: https://visualstudio.microsoft.com/ +.. _PCBuild readme: https://github.com/python/cpython/blob/main/PCbuild/readme.txt +.. _Git for Windows download from the official Git website: https://git-scm.com/download/win .. _build-dependencies: From 141c2d658253897f8a9548affa8ac2927cbfe891 Mon Sep 17 00:00:00 2001 From: Lance Goyke Date: Mon, 20 Nov 2023 09:26:33 -0600 Subject: [PATCH 7/8] added getOS() listener -- oops! --- getting-started/setup-building.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst index 4f19aabdf..b4b3c7c37 100644 --- a/getting-started/setup-building.rst +++ b/getting-started/setup-building.rst @@ -5,6 +5,14 @@ Setup and building ================== +.. raw:: html + + + .. highlight:: console These instructions cover how to get a working copy of the source code and a From ed82d4ed2c3204f9796fd117860e398045258521 Mon Sep 17 00:00:00 2001 From: Lance Goyke Date: Mon, 20 Nov 2023 09:27:49 -0600 Subject: [PATCH 8/8] remove repetitive note about windows --- getting-started/setup-building.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/getting-started/setup-building.rst b/getting-started/setup-building.rst index b4b3c7c37..7a2cf702a 100644 --- a/getting-started/setup-building.rst +++ b/getting-started/setup-building.rst @@ -354,8 +354,7 @@ Install dependencies ==================== This section explains how to install additional extensions (e.g. ``zlib``) -on :ref:`Linux ` and :ref:`macOS`. On Windows, -extensions are already included and built automatically. +on :ref:`Linux ` and :ref:`macOS`. .. tab:: Linux