From 5dfaa46089cd5b097bb467e87daeacf4fa7bef6a Mon Sep 17 00:00:00 2001 From: YR Chen Date: Fri, 25 Feb 2022 18:23:44 +0800 Subject: [PATCH 01/25] [WindowsBuild] Update host toolchain version --- docs/WindowsBuild.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index 4506701290daf..c42a883a77dfc 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -1,8 +1,8 @@ # Building Swift on Windows -Visual Studio 2017 or newer is needed to build Swift on Windows. The free Community edition is sufficient to build Swift. +Visual Studio 2017 or newer is needed to build Swift on Windows, while VS2019 is recommended and currently used for CI. The free Community edition is sufficient to build Swift. -The commands below (with the exception of installing Visual Studio) must be entered in the "**x64 Native** Tools Command Prompt for VS2017" (or VS2019, VS2019 Preview depending on the Visual Studio that you are using) in the Start Menu. This sets environment variables to select the correct target platform. +The commands below (with the exception of installing Visual Studio) must be entered in the "**x64 Native** Tools Command Prompt for VS2019" (or VS2017, VS2022 depending on the Visual Studio that you are using) in the Start Menu. This sets environment variables to select the correct target platform. ## Install dependencies @@ -18,8 +18,7 @@ vs_community ^ --add Microsoft.VisualStudio.Component.VC.ATL ^ --add Microsoft.VisualStudio.Component.VC.CMake.Project ^ --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 ^ - --add Microsoft.VisualStudio.Component.Windows10SDK ^ - --add Microsoft.VisualStudio.Component.Windows10SDK.17763 + --add Microsoft.VisualStudio.Component.Windows10SDK.19041 del /q vs_community.exe ``` @@ -29,12 +28,12 @@ The following [link](https://docs.microsoft.com/visualstudio/install/workload-co ### Python -The command above already installs Python 3. Alternatively, in the Visual Studio installation program, under *Individual Components*, install *Python 3 64 bits (3.7.x)*. +The command above already installs Python 3. Alternatively, in the Visual Studio installation program, under *Individual Components*, install *Python 3 64 bits (3.9.x)*. If you are building a debug version of Swift, you should also install the Python debug binaries. 1. In the Windows settings, go to *Add and Remove Programs* -2. Select the *Python 3.7.x (64-bit)* entry +2. Select the *Python 3.9.x (64-bit)* entry 3. Click *Modify*, then *Yes*, then *Modify* again and then *Next* 4. Select *Download debug binaries (requires VS 2015 or later)* 5. Click *Install* From b8985d3db520ff0d391780a2eb198f92d43928c2 Mon Sep 17 00:00:00 2001 From: YR Chen Date: Fri, 25 Feb 2022 18:37:21 +0800 Subject: [PATCH 02/25] [WindowsBuild] Fix notes --- docs/WindowsBuild.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index c42a883a77dfc..6bac70d3d31c3 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -105,6 +105,8 @@ Set up the `ucrt`, `visualc`, and `WinSDK` modules by: - copying `winsdk.modulemap` located at `swift/stdlib/public/Platform/winsdk.modulemap` into `${UniversalCRTSdkDir}/Include/${UCRTVersion}/um` - and setup the `visualc.apinotes` located at `swift/stdlib/public/Platform/visualc.apinotes` into `${VCToolsInstallDir}/include` as `visualc.apinotes` +You're recommended to create symbolic links to avoid manual copying every time you modified these files. + ```cmd mklink "%UniversalCRTSdkDir%\Include\%UCRTVersion%\ucrt\module.modulemap" S:\swift\stdlib\public\Platform\ucrt.modulemap mklink "%UniversalCRTSdkDir%\Include\%UCRTVersion%\um\module.modulemap" S:\swift\stdlib\public\Platform\winsdk.modulemap @@ -112,7 +114,7 @@ mklink "%VCToolsInstallDir%\include\module.modulemap" S:\swift\stdlib\public\Pla mklink "%VCToolsInstallDir%\include\visualc.apinotes" S:\swift\stdlib\public\Platform\visualc.apinotes ``` -Warning: Creating the above links usually requires administrator privileges. The quick and easy way to do this is to open a second developer prompt by right clicking whatever shortcut you used to open the first one, choosing Run As Administrator, and pasting the above commands into the resulting window. You can then close the privileged prompt; this is the only step which requires elevation. +> **WARNING:** Creating the above links usually requires administrator privileges. The quick and easy way to do this is to open a second developer prompt by right clicking whatever shortcut you used to open the first one, choosing "More > Run As Administrator", and pasting the above commands into the resulting window. You can then close the privileged prompt; this is the only step which requires elevation. ## Build the toolchain @@ -127,7 +129,7 @@ cmake -B "S:\b\1" ^ -D CMAKE_CXX_FLAGS="/GS- /Oy /Gw /Gy" ^ -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ -D CMAKE_MT=mt ^ - -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ -D LLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-windows-msvc ^ -D LLVM_ENABLE_PDB=YES ^ -D LLVM_EXTERNAL_CMARK_SOURCE_DIR=S:\cmark ^ From b94f04bd8bb7ad208b815fe856020408f597c21b Mon Sep 17 00:00:00 2001 From: YR Chen Date: Fri, 25 Feb 2022 20:31:51 +0800 Subject: [PATCH 03/25] [WindowsBuild] Use update-checkout --- docs/WindowsBuild.md | 30 ++++++++++-------------------- 1 file changed, 10 insertions(+), 20 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index 6bac70d3d31c3..563b3cf2af480 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -4,6 +4,8 @@ Visual Studio 2017 or newer is needed to build Swift on Windows, while VS2019 is The commands below (with the exception of installing Visual Studio) must be entered in the "**x64 Native** Tools Command Prompt for VS2019" (or VS2017, VS2022 depending on the Visual Studio that you are using) in the Start Menu. This sets environment variables to select the correct target platform. +> **NOTE:** This guide is intended for toolchain developers who wants to develop or build Swift on their own machine. For building and packaging a standard toolchain, please refer to [`build-windows-toolchain.bat`](../utils/build-windows-toolchain.bat). + ## Install dependencies ### Visual Studio @@ -44,32 +46,20 @@ From the settings application, go to `Update & Security`. In the `For developer ## Clone the repositories -1. Clone `swift/main` branch of `apple/llvm-project` into the build workspace -2. Clone `apple/swift-cmark`, `apple/swift`, `apple/swift-corelibs-libdispatch`, `apple/swift-corelibs-foundation`, `apple/swift-corelibs-xctest`, `apple/swift-tools-support-core`, `apple/swift-llbuild`, `apple/swift-argument-parser`, `apple/swift-driver`, `apple/swift-package-manager`, `JPSim/Yams`, `apple/indexstore-db` into the build workspace - -- Currently, other repositories in the Swift project have not been tested and may not be supported. +> **NOTE:** This guide assumes your sources live at the root of `S:`. If your sources live elsewhere, you can create a substitution for this: +> subst S: -This guide assumes your sources live at the root of `S:`. If your sources live elsewhere, you can create a substitution for this: +First, clone `apple/swift` (this repository) with Git: ```cmd -subst S: +S: +git clone -c core.autocrlf=input -c core.symlinks=true https://github.com/apple/swift ``` +You'll be able to clone and check out the rest of Swift source repositories with `update-checkout` tool: + ```cmd -S: -git clone https://github.com/apple/llvm-project --branch swift/main llvm-project -git clone -c core.autocrlf=input -c core.symlinks=true https://github.com/apple/swift swift -git clone https://github.com/apple/swift-cmark cmark -git clone https://github.com/apple/swift-corelibs-libdispatch swift-corelibs-libdispatch -git clone https://github.com/apple/swift-corelibs-foundation swift-corelibs-foundation -git clone https://github.com/apple/swift-corelibs-xctest swift-corelibs-xctest -git clone https://github.com/apple/swift-tools-support-core swift-tools-support-core -git clone -c core.symlinks=true https://github.com/apple/swift-llbuild swift-llbuild -git clone https://github.com/JPSim/Yams Yams -git clone https://github.com/apple/swift-driver swift-driver -git clone https://github.com/apple/swift-argument-parser swift-argument-parser -git clone -c core.autocrlf=input https://github.com/apple/swift-package-manager swift-package-manager -git clone https://github.com/apple/indexstore-db indexstore-db +swift\utils\update-checkout.cmd --clone ``` ## Dependencies (ICU, SQLite3, curl, libxml2 and zlib) From 2cd456579638adb69a1812b1053bf33e13fdc32a Mon Sep 17 00:00:00 2001 From: YR Chen Date: Fri, 25 Feb 2022 22:45:57 +0800 Subject: [PATCH 04/25] [WindowsBuild] Adjust paragraph structure --- docs/WindowsBuild.md | 108 ++++++++++++++++++++++++++++------------ utils/build-windows.bat | 2 +- 2 files changed, 77 insertions(+), 33 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index 563b3cf2af480..c368c631cf528 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -1,10 +1,10 @@ # Building Swift on Windows -Visual Studio 2017 or newer is needed to build Swift on Windows, while VS2019 is recommended and currently used for CI. The free Community edition is sufficient to build Swift. +Visual Studio 2017 or newer is needed to build Swift on Windows, while VS2019 is recommended and currently used for CI. The free Community edition is sufficient to build Swift, and we're assuming host and target to be both x64. The commands below (with the exception of installing Visual Studio) must be entered in the "**x64 Native** Tools Command Prompt for VS2019" (or VS2017, VS2022 depending on the Visual Studio that you are using) in the Start Menu. This sets environment variables to select the correct target platform. -> **NOTE:** This guide is intended for toolchain developers who wants to develop or build Swift on their own machine. For building and packaging a standard toolchain, please refer to [`build-windows-toolchain.bat`](../utils/build-windows-toolchain.bat). +> **NOTE:** This guide is intended for toolchain developers who wants to develop or build Swift on their own machine. For building a standard toolchain, please refer to [`build-windows-toolchain.bat`](../utils/build-windows-toolchain.bat). ## Install dependencies @@ -85,7 +85,7 @@ Note that ICU is only required for building Foundation, and SQLite is only needed for building llbuild and onwards. The ICU project provides binaries, alternatively, see the ICU project for details on building ICU from source. -## One-time Setup (re-run on Visual Studio upgrades) +## One-time setup (re-run on Visual Studio upgrades) Set up the `ucrt`, `visualc`, and `WinSDK` modules by: @@ -106,41 +106,82 @@ mklink "%VCToolsInstallDir%\include\visualc.apinotes" S:\swift\stdlib\public\Pla > **WARNING:** Creating the above links usually requires administrator privileges. The quick and easy way to do this is to open a second developer prompt by right clicking whatever shortcut you used to open the first one, choosing "More > Run As Administrator", and pasting the above commands into the resulting window. You can then close the privileged prompt; this is the only step which requires elevation. -## Build the toolchain +## Build a minimal Swift toolchain + +A minimal Swift toolchain comes with neither optional features nor SwiftPM stuffs, and is useful for playing with the compiler or language itself. ```cmd -cmake -B "S:\b\1" ^ +cmake -B S:\b\1 ^ -C S:\swift\cmake\caches\Windows-x86_64.cmake ^ -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr ^ -D CMAKE_C_COMPILER=cl ^ - -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ + -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy /source-charset:utf-8 /execution-charset:utf-8" ^ -D CMAKE_CXX_COMPILER=cl ^ - -D CMAKE_CXX_FLAGS="/GS- /Oy /Gw /Gy" ^ + -D CMAKE_CXX_FLAGS="/GS- /Oy /Gw /Gy /utf-8" ^ + -D CMAKE_MT=mt ^ -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D LLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-windows-msvc ^ + -D LLVM_APPEND_VC_REV=NO ^ + -D LLVM_EXTERNAL_CMARK_SOURCE_DIR=S:\cmark ^ + -D LLVM_EXTERNAL_SWIFT_SOURCE_DIR=S:\swift ^ + -D SWIFT_PATH_TO_LIBDISPATCH_SOURCE=S:\swift-corelibs-libdispatch ^ + -G Ninja ^ + -S S:\llvm-project\llvm + +cmake --build S:\b\1 +cmake --build S:\b\1 --target install +``` + +## Build a complete toolchain for development + +The following guide will get you through the building process of a complete Swift debug toolchain. + +### Build Swift + +```cmd +cmake -B S:\b\1 ^ + -C S:\swift\cmake\caches\Windows-x86_64.cmake ^ + -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ + -D CMAKE_C_COMPILER=cl ^ + -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy /source-charset:utf-8 /execution-charset:utf-8" ^ + -D CMAKE_CXX_COMPILER=cl ^ + -D CMAKE_CXX_FLAGS="/GS- /Oy /Gw /Gy /utf-8" ^ -D CMAKE_MT=mt ^ + -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ -D LLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-windows-msvc ^ -D LLVM_ENABLE_PDB=YES ^ + -D LLVM_EXTERNAL_CMARK_SOURCE_DIR=S:\cmark ^ -D LLVM_EXTERNAL_SWIFT_SOURCE_DIR=S:\swift ^ -D SWIFT_PATH_TO_LIBDISPATCH_SOURCE=S:\swift-corelibs-libdispatch ^ + + -D SWIFT_ENABLE_EXPERIMENTAL_CONCURRENCY=YES ^ + -D SWIFT_ENABLE_EXPERIMENTAL_DISTRIBUTED=YES ^ + -D SWIFT_ENABLE_EXPERIMENTAL_DIFFERENTIABLE_PROGRAMMING=YES ^ + + -D SWIFT_ENABLE_EXPERIMENTAL_STRING_PROCESSING=YES ^ + -D EXPERIMENTAL_STRING_PROCESSING_SOURCE_DIR=S:\swift-experimental-string-processing ^ + -G Ninja ^ -S S:\llvm-project\llvm -ninja -C S:\b\1 +cmake --build S:\b\1 ``` > **NOTE:** Linking with debug information (`-D LLVM_ENABLE_PDB=YES`) is very memory intensive. When building with parallel jobs, it is possible to consume upwards of 32 GiB of RAM. You can append `-D LLVM_PARALLEL_LINK_JOBS=N -D DLLVM_PARALLEL_LINK_JOBS=N` to reduce the number of parallel link operations to `N` which should help reduce the memory pressure. You may need to set this to a low number (e.g. 1) if you see build failures due to memory exhaustion. -## Running Swift tests on Windows +Test Swift: ```cmd path S:\Library\icu-67\usr\bin;S:\b\1\bin;S:\b\1\tools\swift\libdispatch-windows-x86_64-prefix\bin;%PATH%;%ProgramFiles%\Git\usr\bin ninja -C S:\b\1 check-swift ``` -## Build swift-corelibs-libdispatch +### Build libdispatch ```cmd cmake -B S:\b\2 ^ @@ -157,13 +198,13 @@ cmake -B S:\b\2 ^ ninja -C S:\b\2 ``` -## Test swift-corelibs-libdispatch +Test libdispatch: ```cmd ninja -C S:\b\2 check ``` -## Build swift-corelibs-foundation +### Build Foundation ```cmd cmake -B S:\b\3 ^ @@ -188,13 +229,13 @@ cmake -B S:\b\3 ^ ninja -C S:\b\3 ``` -- Add Foundation to your path: +Add Foundation to your path: ```cmd path S:\b\3\bin;%PATH% ``` -## Build swift-corelibs-xctest +### Build XCTest ```cmd cmake -B S:\b\4 ^ @@ -211,20 +252,20 @@ cmake -B S:\b\4 ^ ninja -C S:\b\4 ``` -- Add XCTest to your path: +Add XCTest to your path: ```cmd path S:\b\4;%PATH% ``` -## Test XCTest +Test XCTest: ```cmd ninja -C S:\b\4 check-xctest ``` -## Rebuild Foundation +### Rebuild Foundation ```cmd cmake -B S:\b\3 ^ @@ -250,13 +291,13 @@ cmake -B S:\b\3 ^ ninja -C S:\b\3 ``` -## Test Foundation +Test Foundation: ```cmd ninja -C S:\b\3 test ``` -## Build swift-tools-core-support +### Build TSC ```cmd cmake -B S:\b\5 ^ @@ -275,7 +316,7 @@ cmake -B S:\b\5 ^ ninja -C S:\b\5 ``` -## Build swift-llbuild +### Build llbuild ```cmd cmake -B S:\b\6 ^ @@ -297,13 +338,13 @@ cmake -B S:\b\6 ^ ninja -C S:\b\6 ``` -- Add llbuild to your path: +Add llbuild to your path: ```cmd path S:\b\6\bin;%PATH% ``` -## Build Yams +### Build Yams ```cmd cmake -B S:\b\7 ^ @@ -321,7 +362,7 @@ cmake -B S:\b\7 ^ ninja -C S:\b\7 ``` -## Build swift-argument-parser +### Build ArgumentParser ```cmd cmake -B S:\b\8 ^ @@ -339,7 +380,7 @@ cmake -B S:\b\8 ^ ninja -C S:\b\8 ``` -## Build swift-driver +## Build SwiftDriver ```cmd cmake -B S:\b\9 ^ @@ -360,7 +401,7 @@ cmake -B S:\b\9 ^ ninja -C S:\b\9 ``` -## Build swift-package-manager +### Build SwiftPM ```cmd cmake -B S:\b\10 ^ @@ -378,22 +419,25 @@ cmake -B S:\b\10 ^ -D ArgumentParser_DIR=S:\b\8\cmake\modules ^ -D SwiftDriver_DIR=S:\b\9\cmake\modules ^ -G Ninja ^ - -S S:\swift-package-manager + -S S:\swiftpm ninja -C S:\b\10 ``` -Indicate to swift-package-manager where to find the Package Description before installation: +Indicate to SwiftPM where to find `PackageDescription` before installation: + ```cmd set SWIFTPM_PD_LIBS=S:\b\10\pm ``` -## Install the Swift toolchain on Windows - -- Run ninja install: +### Install the toolchain ```cmd -ninja -C S:\b\1 install +cmake --build S:\b\1 --target install ``` -- Add the Swift on Windows binaries path (`C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\bin`) to the `PATH` environment variable. +Add the target to path: + +```cmd +path S:\b\toolchain\usr\bin:%PATH% +``` diff --git a/utils/build-windows.bat b/utils/build-windows.bat index fc82212772c33..d9167d53eff84 100644 --- a/utils/build-windows.bat +++ b/utils/build-windows.bat @@ -20,7 +20,7 @@ :: Practically, it is easier to be in the Adminstrators group to run the :: script, but it should be possible to execute as a normal user. :: The user will need permission to write files into the Windows SDK and the -:: VisualC++ folder. +:: Visual C++ folder. :: @echo off From de18b55082aadace2c78ceee7a3b7543424e82bd Mon Sep 17 00:00:00 2001 From: YR Chen Date: Sat, 26 Feb 2022 20:21:45 +0800 Subject: [PATCH 05/25] [WindowsBuild] No longer suggest enabling PDB for Swift --- docs/WindowsBuild.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index c368c631cf528..45273ff4255a9 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -134,7 +134,7 @@ cmake --build S:\b\1 cmake --build S:\b\1 --target install ``` -## Build a complete toolchain for development +## Build a complete toolchain with debugging info The following guide will get you through the building process of a complete Swift debug toolchain. @@ -143,7 +143,7 @@ The following guide will get you through the building process of a complete Swif ```cmd cmake -B S:\b\1 ^ -C S:\swift\cmake\caches\Windows-x86_64.cmake ^ - -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ -D CMAKE_C_COMPILER=cl ^ -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy /source-charset:utf-8 /execution-charset:utf-8" ^ @@ -153,7 +153,6 @@ cmake -B S:\b\1 ^ -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ -D LLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-windows-msvc ^ - -D LLVM_ENABLE_PDB=YES ^ -D LLVM_EXTERNAL_CMARK_SOURCE_DIR=S:\cmark ^ -D LLVM_EXTERNAL_SWIFT_SOURCE_DIR=S:\swift ^ @@ -172,7 +171,9 @@ cmake -B S:\b\1 ^ cmake --build S:\b\1 ``` -> **NOTE:** Linking with debug information (`-D LLVM_ENABLE_PDB=YES`) is very memory intensive. When building with parallel jobs, it is possible to consume upwards of 32 GiB of RAM. You can append `-D LLVM_PARALLEL_LINK_JOBS=N -D DLLVM_PARALLEL_LINK_JOBS=N` to reduce the number of parallel link operations to `N` which should help reduce the memory pressure. You may need to set this to a low number (e.g. 1) if you see build failures due to memory exhaustion. +> **NOTE:** If you want to profile the Swift compiler, you may need to link it with debug information. You can enable this by specifying `-D LLVM_ENABLE_PDB=YES`. +> +> Linking with debug information is very memory-intensive and may drastically slow down the linking process. A single link job is possible to consume upwards of 10 GiB of RAM. You can append `-D LLVM_PARALLEL_LINK_JOBS=N` to reduce the number of parallel link operations to `N` which should help reduce the memory pressure. Test Swift: From e53a88d80433c704570f5f57bd8487355e18b28a Mon Sep 17 00:00:00 2001 From: YR Chen Date: Sat, 26 Feb 2022 23:54:41 +0800 Subject: [PATCH 06/25] [WindowsBuild] Guide up to libdispatch --- docs/WindowsBuild.md | 101 ++++++++++++++++++++++++++++++++++--------- 1 file changed, 81 insertions(+), 20 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index 45273ff4255a9..834bf990f2ea1 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -64,7 +64,7 @@ swift\utils\update-checkout.cmd --clone ## Dependencies (ICU, SQLite3, curl, libxml2 and zlib) -The instructions assume that the dependencies are in `S:/Library`. The directory +The instructions assume that the dependencies are in `S:\Library`. The directory structure should resemble: ``` @@ -138,13 +138,14 @@ cmake --build S:\b\1 --target install The following guide will get you through the building process of a complete Swift debug toolchain. -### Build Swift +### Swift compiler ```cmd cmake -B S:\b\1 ^ -C S:\swift\cmake\caches\Windows-x86_64.cmake ^ -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ + -D CMAKE_C_COMPILER=cl ^ -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy /source-charset:utf-8 /execution-charset:utf-8" ^ -D CMAKE_CXX_COMPILER=cl ^ @@ -152,8 +153,8 @@ cmake -B S:\b\1 ^ -D CMAKE_MT=mt ^ -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ - -D LLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-windows-msvc ^ + -D LLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-windows-msvc ^ -D LLVM_EXTERNAL_CMARK_SOURCE_DIR=S:\cmark ^ -D LLVM_EXTERNAL_SWIFT_SOURCE_DIR=S:\swift ^ -D SWIFT_PATH_TO_LIBDISPATCH_SOURCE=S:\swift-corelibs-libdispatch ^ @@ -175,37 +176,80 @@ cmake --build S:\b\1 > > Linking with debug information is very memory-intensive and may drastically slow down the linking process. A single link job is possible to consume upwards of 10 GiB of RAM. You can append `-D LLVM_PARALLEL_LINK_JOBS=N` to reduce the number of parallel link operations to `N` which should help reduce the memory pressure. +> **NOTE:** By default, we enables all the experimental features in Swift by `-D SWIFT_ENABLE_EXPERIMENTAL_{FEATURE}=YES`. These features can be disabled separately. Notice that `Concurrency` is an accepted language feature that should be enabled for Swift 5.5+. + Test Swift: ```cmd -path S:\Library\icu-67\usr\bin;S:\b\1\bin;S:\b\1\tools\swift\libdispatch-windows-x86_64-prefix\bin;%PATH%;%ProgramFiles%\Git\usr\bin -ninja -C S:\b\1 check-swift +path S:\b\1\bin;S:\b\1\tools\swift\libdispatch-windows-x86_64-prefix\bin;%PATH% +cmake --build S:\b\1 --target check-swift ``` -### Build libdispatch +### Swift Standard Library ```cmd cmake -B S:\b\2 ^ + -C S:\swift\cmake\caches\Runtime-Windows-x86_64.cmake ^ -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ - -D CMAKE_INSTALL_PREFIX=C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr ^ + -D CMAKE_INSTALL_PREFIX=S:\b\sdk\usr ^ + -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ -D CMAKE_CXX_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_CXX_FLAGS="/GS- /Oy /Gw /Gy" ^ -D CMAKE_MT=mt ^ - -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ + -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ + + -D LLVM_DIR=S:\b\1\lib\cmake\llvm ^ + -D SWIFT_NATIVE_SWIFT_TOOLS_PATH=S:\b\1\bin ^ + -D SWIFT_PATH_TO_LIBDISPATCH_SOURCE=S:\swift-corelibs-libdispatch ^ + + -D SWIFT_ENABLE_EXPERIMENTAL_CONCURRENCY=YES ^ + -D SWIFT_ENABLE_EXPERIMENTAL_DISTRIBUTED=YES ^ + -D SWIFT_ENABLE_EXPERIMENTAL_DIFFERENTIABLE_PROGRAMMING=YES ^ + + -D SWIFT_ENABLE_EXPERIMENTAL_STRING_PROCESSING=YES ^ + -D EXPERIMENTAL_STRING_PROCESSING_SOURCE_DIR=S:\swift-experimental-string-processing ^ + + -G Ninja ^ + -S S:\swift + +cmake --build S:\b\2 +``` + +> **NOTE:** Swift Standard Library is also built along with the compiler. This step extracts it into a portable SDK where we will install other runtime libraries. + +### libdispatch + +```cmd +cmake -B S:\b\3 ^ + -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_INSTALL_PREFIX=S:\b\sdk\usr ^ + -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ + -D CMAKE_CXX_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_CXX_FLAGS="/GS- /Oy /Gw /Gy" ^ + -D CMAKE_MT=mt ^ + -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D ENABLE_SWIFT=YES ^ + -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ + -G Ninja ^ -S S:\swift-corelibs-libdispatch -ninja -C S:\b\2 +ninja -C S:\b\3 ``` Test libdispatch: ```cmd -ninja -C S:\b\2 check +ninja -C S:\b\3 check ``` -### Build Foundation +### Foundation ```cmd cmake -B S:\b\3 ^ @@ -236,7 +280,7 @@ Add Foundation to your path: path S:\b\3\bin;%PATH% ``` -### Build XCTest +### XCTest ```cmd cmake -B S:\b\4 ^ @@ -298,7 +342,7 @@ Test Foundation: ninja -C S:\b\3 test ``` -### Build TSC +### TSC ```cmd cmake -B S:\b\5 ^ @@ -317,7 +361,7 @@ cmake -B S:\b\5 ^ ninja -C S:\b\5 ``` -### Build llbuild +### llbuild ```cmd cmake -B S:\b\6 ^ @@ -345,7 +389,7 @@ Add llbuild to your path: path S:\b\6\bin;%PATH% ``` -### Build Yams +### Yams ```cmd cmake -B S:\b\7 ^ @@ -363,7 +407,7 @@ cmake -B S:\b\7 ^ ninja -C S:\b\7 ``` -### Build ArgumentParser +### ArgumentParser ```cmd cmake -B S:\b\8 ^ @@ -381,7 +425,7 @@ cmake -B S:\b\8 ^ ninja -C S:\b\8 ``` -## Build SwiftDriver +### SwiftDriver ```cmd cmake -B S:\b\9 ^ @@ -402,7 +446,7 @@ cmake -B S:\b\9 ^ ninja -C S:\b\9 ``` -### Build SwiftPM +### SwiftPM ```cmd cmake -B S:\b\10 ^ @@ -431,14 +475,31 @@ Indicate to SwiftPM where to find `PackageDescription` before installation: set SWIFTPM_PD_LIBS=S:\b\10\pm ``` -### Install the toolchain +## Gather the toolchain and SDK + +If you want a toolchain for real-world testing or distribution, you can use CMake to perform the install step. + +### Swift compiler and standard library ```cmd cmake --build S:\b\1 --target install ``` -Add the target to path: +For testing, add the target to path: ```cmd path S:\b\toolchain\usr\bin:%PATH% ``` + +## Swift Windows SDK (with core libraries) + +```cmd +cmake --build S:\b\2 --target install +cmake --build S:\b\3 --target install +``` + +For testing, set `SDKROOT`: + +```cmd +set SDKROOT=S:\b\sdk +``` From 7e05448900bfc383250ddf52cd4d56d9468863ab Mon Sep 17 00:00:00 2001 From: YR Chen Date: Sun, 27 Feb 2022 01:43:50 +0800 Subject: [PATCH 07/25] [WindowsBuild] Use `vcpkg` for Foundation --- docs/WindowsBuild.md | 69 ++++++++++++++++++-------------------------- 1 file changed, 28 insertions(+), 41 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index 834bf990f2ea1..94cf93ec3f7c5 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -62,30 +62,18 @@ You'll be able to clone and check out the rest of Swift source repositories with swift\utils\update-checkout.cmd --clone ``` -## Dependencies (ICU, SQLite3, curl, libxml2 and zlib) +## Set up `vcpkg` -The instructions assume that the dependencies are in `S:\Library`. The directory -structure should resemble: +The instructions will use `vcpkg` for pulling in external dependencies, including ICU, libcurl, libxml2, SQLite 3 and zlib. -``` -/Library - ┝ icu-67 - │ ┕ usr/... - ├ libcurl-development - │ ┕ usr/... - ├ libxml2-development - │ ┕ usr/... - ├ sqlite-3.28.0 - │ ┕ usr/... - ┕ zlib-1.2.11 - ┕ usr/... -``` +Before you get started, clone and bootstrap `vcpkg`: -Note that ICU is only required for building Foundation, and SQLite is only -needed for building llbuild and onwards. The ICU project provides binaries, -alternatively, see the ICU project for details on building ICU from source. +```cmd +git clone https://github.com/microsoft/vcpkg +vcpkg\bootstrap-vcpkg.bat +``` -## One-time setup (re-run on Visual Studio upgrades) +## Set up Visual Studio integration (re-run on Visual Studio upgrades) Set up the `ucrt`, `visualc`, and `WinSDK` modules by: @@ -134,7 +122,7 @@ cmake --build S:\b\1 cmake --build S:\b\1 --target install ``` -## Build a complete toolchain with debugging info +## Build a complete toolchain for development The following guide will get you through the building process of a complete Swift debug toolchain. @@ -226,6 +214,7 @@ cmake --build S:\b\2 cmake -B S:\b\3 ^ -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ -D CMAKE_INSTALL_PREFIX=S:\b\sdk\usr ^ + -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ -D CMAKE_CXX_COMPILER=S:/b/1/bin/clang-cl.exe ^ @@ -240,44 +229,41 @@ cmake -B S:\b\3 ^ -G Ninja ^ -S S:\swift-corelibs-libdispatch -ninja -C S:\b\3 +cmake --build S:\b\3 ``` Test libdispatch: ```cmd -ninja -C S:\b\3 check +cmake --build S:\b\3 check ``` ### Foundation ```cmd -cmake -B S:\b\3 ^ +cmake -B S:\b\4 ^ -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ - -D CMAKE_INSTALL_PREFIX=C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr ^ + -D CMAKE_INSTALL_PREFIX=S:\b\sdk\usr ^ + -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ + -D CMAKE_CXX_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_CXX_FLAGS="/GS- /Oy /Gw /Gy" ^ -D CMAKE_MT=mt ^ + -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ + + -D ENABLE_SWIFT=YES ^ -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ - -D CURL_LIBRARY="S:/Library/libcurl-development/usr/lib/libcurl.lib" ^ - -D CURL_INCLUDE_DIR="S:/Library/libcurl-development/usr/include" ^ - -D ICU_I18N_LIBRARY_RELEASE=S:\library\icu-67\usr\lib\icuin67.lib ^ - -D ICU_ROOT=S:\Library\icu-67\usr ^ - -D ICU_UC_LIBRARY_RELEASE=S:\Library\icu-67\usr\lib\icuuc67.lib ^ - -D LIBXML2_DEFINITIONS="/DLIBXML_STATIC" ^ - -D LIBXML2_LIBRARY=S:\Library\libxml2-development\usr\lib\libxml2s.lib ^ - -D LIBXML2_INCLUDE_DIR=S:\Library\libxml2-development\usr\include\libxml2 ^ + -D dispatch_DIR=S:\b\3\cmake\modules ^ + + -D CMAKE_TOOLCHAIN_FILE=S:\vcpkg\scripts\buildsystems\vcpkg.cmake ^ + -D ENABLE_TESTING=NO ^ - -D dispatch_DIR=S:\b\2\cmake\modules ^ -G Ninja ^ -S S:\swift-corelibs-foundation -ninja -C S:\b\3 -``` - -Add Foundation to your path: - -```cmd -path S:\b\3\bin;%PATH% +cmake --build S:\b\4 ``` ### XCTest @@ -496,6 +482,7 @@ path S:\b\toolchain\usr\bin:%PATH% ```cmd cmake --build S:\b\2 --target install cmake --build S:\b\3 --target install +cmake --build S:\b\4 --target install ``` For testing, set `SDKROOT`: From dc271911a50d424e05af09417f4bbb9982555109 Mon Sep 17 00:00:00 2001 From: YR Chen Date: Sun, 27 Feb 2022 02:33:28 +0800 Subject: [PATCH 08/25] [WindowsBuild] Fix Swift and libdispatch tests --- docs/WindowsBuild.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index 94cf93ec3f7c5..1f974c92fd117 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -143,6 +143,7 @@ cmake -B S:\b\1 ^ -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ -D LLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-windows-msvc ^ + -D LLVM_EXTERNAL_CMARK_SOURCE_DIR=S:\cmark ^ -D LLVM_EXTERNAL_SWIFT_SOURCE_DIR=S:\swift ^ -D SWIFT_PATH_TO_LIBDISPATCH_SOURCE=S:\swift-corelibs-libdispatch ^ @@ -169,7 +170,7 @@ cmake --build S:\b\1 Test Swift: ```cmd -path S:\b\1\bin;S:\b\1\tools\swift\libdispatch-windows-x86_64-prefix\bin;%PATH% +path %PATH%;%ProgramFiles%\Git\usr\bin cmake --build S:\b\1 --target check-swift ``` @@ -226,6 +227,8 @@ cmake -B S:\b\3 ^ -D ENABLE_SWIFT=YES ^ -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ + -D BUILD_TESTING=YES ^ + -G Ninja ^ -S S:\swift-corelibs-libdispatch @@ -235,7 +238,7 @@ cmake --build S:\b\3 Test libdispatch: ```cmd -cmake --build S:\b\3 check +cmake --build S:\b\3 --target test ``` ### Foundation From 2a1e78e0c14f22dbd4417e54177a31e09984981e Mon Sep 17 00:00:00 2001 From: YR Chen Date: Sun, 27 Feb 2022 14:35:51 +0800 Subject: [PATCH 09/25] [WindowsBuild] XCTest and TestFoundation --- docs/WindowsBuild.md | 73 ++++++++++++++++++++++++-------------------- 1 file changed, 40 insertions(+), 33 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index 1f974c92fd117..49508d7c0a9cd 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -126,6 +126,12 @@ cmake --build S:\b\1 --target install The following guide will get you through the building process of a complete Swift debug toolchain. +Before you kick off the process, unset `SDKROOT` if you've already installed Swift: + +```cmd +set SDKROOT= +``` + ### Swift compiler ```cmd @@ -241,7 +247,7 @@ Test libdispatch: cmake --build S:\b\3 --target test ``` -### Foundation +### Foundation (without tests) ```cmd cmake -B S:\b\4 ^ @@ -250,19 +256,15 @@ cmake -B S:\b\4 ^ -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ - -D CMAKE_CXX_COMPILER=S:/b/1/bin/clang-cl.exe ^ - -D CMAKE_CXX_FLAGS="/GS- /Oy /Gw /Gy" ^ -D CMAKE_MT=mt ^ -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ - -D ENABLE_SWIFT=YES ^ -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ -D dispatch_DIR=S:\b\3\cmake\modules ^ -D CMAKE_TOOLCHAIN_FILE=S:\vcpkg\scripts\buildsystems\vcpkg.cmake ^ - -D ENABLE_TESTING=NO ^ -G Ninja ^ -S S:\swift-corelibs-foundation @@ -272,63 +274,68 @@ cmake --build S:\b\4 ### XCTest ```cmd -cmake -B S:\b\4 ^ +cmake -B S:\b\5 ^ -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ - -D CMAKE_INSTALL_PREFIX=C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr ^ + -D CMAKE_INSTALL_PREFIX=S:\b\sdk\usr ^ + -D CMAKE_MT=mt ^ + -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ - -D dispatch_DIR=S:\b\2\cmake\modules ^ - -D Foundation_DIR=S:\b\3\cmake\modules ^ + -D dispatch_DIR=S:\b\3\cmake\modules ^ + -D Foundation_DIR=S:\b\4\cmake\modules ^ + + -D ENABLE_TESTING=YES ^ + -D LLVM_DIR=S:\b\1\lib\cmake\llvm ^ -D LIT_COMMAND=S:\llvm-project\llvm\utils\lit\lit.py ^ + -D XCTEST_PATH_TO_LIBDISPATCH_SOURCE=S:\swift-corelibs-libdispatch ^ + -D XCTEST_PATH_TO_LIBDISPATCH_BUILD=S:\b\3 ^ + -D XCTEST_PATH_TO_FOUNDATION_BUILD=S:\b\4 ^ + -G Ninja ^ -S S:\swift-corelibs-xctest -ninja -C S:\b\4 -``` - -Add XCTest to your path: - -```cmd - -path S:\b\4;%PATH% +cmake --build S:\b\5 ``` Test XCTest: ```cmd -ninja -C S:\b\4 check-xctest +path %PATH%;%ProgramFiles%\Git\usr\bin +cmake --build S:\b\5 --target check-xctest ``` -### Rebuild Foundation +### Foundation (with tests) ```cmd -cmake -B S:\b\3 ^ +cmake -B S:\b\4 ^ -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ - -D CMAKE_INSTALL_PREFIX=C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr ^ + -D CMAKE_INSTALL_PREFIX=S:\b\sdk\usr ^ + -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ -D CMAKE_MT=mt ^ + -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ - -D CURL_LIBRARY="S:/Library/libcurl-development/usr/lib/libcurl.lib" ^ - -D CURL_INCLUDE_DIR="S:/Library/libcurl-development/usr/include" ^ - -D ICU_I18N_LIBRARY_RELEASE=S:\library\icu-67\usr\lib\icuin67.lib ^ - -D ICU_ROOT=S:\Library\icu-67\usr ^ - -D ICU_UC_LIBRARY_RELEASE=S:\Library\icu-67\usr\lib\icuuc67.lib ^ - -D LIBXML2_LIBRARY=S:\Library\libxml2-development\usr\lib\libxml2s.lib ^ - -D LIBXML2_INCLUDE_DIR=S:\Library\libxml2-development\usr\include\libxml2 ^ - -D LIBXML2_DEFINITIONS="/DLIBXML_STATIC" ^ + -D dispatch_DIR=S:\b\3\cmake\modules ^ + + -D CMAKE_TOOLCHAIN_FILE=S:\vcpkg\scripts\buildsystems\vcpkg.cmake ^ + -D ENABLE_TESTING=YES ^ - -D dispatch_DIR=S:\b\2\cmake\modules ^ - -D XCTest_DIR=S:\b\4\cmake\modules ^ + -D XCTest_DIR=S:\b\5\cmake\modules ^ + -G Ninja ^ -S S:\swift-corelibs-foundation -ninja -C S:\b\3 +cmake --build S:\b\4 ``` Test Foundation: ```cmd -ninja -C S:\b\3 test +cmake --build S:\b\4 --target test ``` ### TSC From 91837c221aa02c3714c03d2d1a020decf637d720 Mon Sep 17 00:00:00 2001 From: stevapple Date: Sun, 27 Feb 2022 15:26:50 +0800 Subject: [PATCH 10/25] [WindowsBuild] Minor fixes --- docs/WindowsBuild.md | 38 ++++++++++++++++++++++++-------------- 1 file changed, 24 insertions(+), 14 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index 49508d7c0a9cd..1ce06e81cbee3 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -2,9 +2,9 @@ Visual Studio 2017 or newer is needed to build Swift on Windows, while VS2019 is recommended and currently used for CI. The free Community edition is sufficient to build Swift, and we're assuming host and target to be both x64. -The commands below (with the exception of installing Visual Studio) must be entered in the "**x64 Native** Tools Command Prompt for VS2019" (or VS2017, VS2022 depending on the Visual Studio that you are using) in the Start Menu. This sets environment variables to select the correct target platform. +The commands below (with the exception of installing Visual Studio) must be entered in the **x64 Native Tools Command Prompt for VS2019** (or VS2017, VS2022 depending on the Visual Studio that you are using) in the Start Menu. This sets environment variables to select the correct target platform. -> **NOTE:** This guide is intended for toolchain developers who wants to develop or build Swift on their own machine. For building a standard toolchain, please refer to [`build-windows-toolchain.bat`](../utils/build-windows-toolchain.bat). +> **NOTE:** This guide is intended for toolchain developers who wants to develop or build Swift on their own machine. For building and packaging a standard toolchain, please refer to [`build-windows-toolchain.bat`](../utils/build-windows-toolchain.bat). ## Install dependencies @@ -24,53 +24,63 @@ vs_community ^ del /q vs_community.exe ``` +> **NOTE:** For anyone who wants to use Visual Studio 2022 instead: +> - replace `https://aka.ms/vs/16/release/vs_community.exe` with `https://aka.ms/vs/17/release/vs_community.exe` +> - replace `Component.CPython3.x64` with `Component.CPython39.x64` + If you prefer you can install everything by hand, but make sure to include "Programming Languages|Visual C++" and "Windows and Web Development|Universal Windows App Development|Windows SDK" in your installation. The components listed above are required. The following [link](https://docs.microsoft.com/visualstudio/install/workload-component-id-vs-build-tools?view=vs-2019) helps in finding the component name given its ID for Visual Studio 2019. ### Python -The command above already installs Python 3. Alternatively, in the Visual Studio installation program, under *Individual Components*, install *Python 3 64 bits (3.9.x)*. +The command above already installs Python 3. Alternatively, in the Visual Studio installation program, under *Individual Components*, install *Python 3 64 bits (3.x.x)*. If you are building a debug version of Swift, you should also install the Python debug binaries. 1. In the Windows settings, go to *Add and Remove Programs* -2. Select the *Python 3.9.x (64-bit)* entry +2. Select the *Python 3.x.x (64-bit)* entry 3. Click *Modify*, then *Yes*, then *Modify* again and then *Next* 4. Select *Download debug binaries (requires VS 2015 or later)* 5. Click *Install* ## Enable Developer Mode -From the settings application, go to `Update & Security`. In the `For developers` tab, select `Developer Mode` for `Use Developer Features`. This is required to enable the creation of symbolic links. +From the settings application, go to *Update & Security*. In the *For developers* tab, select *Developer Mode* for *Use Developer Features*. This is required to enable the creation of symbolic links. ## Clone the repositories > **NOTE:** This guide assumes your sources live at the root of `S:`. If your sources live elsewhere, you can create a substitution for this: -> subst S: +> +> subst S: First, clone `apple/swift` (this repository) with Git: ```cmd -S: -git clone -c core.autocrlf=input -c core.symlinks=true https://github.com/apple/swift +git clone -c core.autocrlf=input -c core.symlinks=true https://github.com/apple/swift S:\swift ``` You'll be able to clone and check out the rest of Swift source repositories with `update-checkout` tool: ```cmd -swift\utils\update-checkout.cmd --clone +S:\swift\utils\update-checkout.cmd --clone ``` ## Set up `vcpkg` -The instructions will use `vcpkg` for pulling in external dependencies, including ICU, libcurl, libxml2, SQLite 3 and zlib. +This guide uses `vcpkg` for pulling in external dependencies, including ICU, libcurl, libxml2, SQLite 3 and zlib. + +All you have to do is to clone and bootstrap `vcpkg`: + +```cmd +git clone https://github.com/microsoft/vcpkg S:\vcpkg +S:\vcpkg\bootstrap-vcpkg.bat +``` -Before you get started, clone and bootstrap `vcpkg`: +By default, the dependencies will be downloaded and built on demand. Optionally, you can prebuild the dependencies in advance: ```cmd -git clone https://github.com/microsoft/vcpkg -vcpkg\bootstrap-vcpkg.bat +S:\vcpkg\vcpkg install curl icu libxml2 sqlite3 zlib --triplet=x64-windows ``` ## Set up Visual Studio integration (re-run on Visual Studio upgrades) @@ -171,7 +181,7 @@ cmake --build S:\b\1 > > Linking with debug information is very memory-intensive and may drastically slow down the linking process. A single link job is possible to consume upwards of 10 GiB of RAM. You can append `-D LLVM_PARALLEL_LINK_JOBS=N` to reduce the number of parallel link operations to `N` which should help reduce the memory pressure. -> **NOTE:** By default, we enables all the experimental features in Swift by `-D SWIFT_ENABLE_EXPERIMENTAL_{FEATURE}=YES`. These features can be disabled separately. Notice that `Concurrency` is an accepted language feature that should be enabled for Swift 5.5+. +> **NOTE:** By default, we enables all the experimental features in Swift by `-D SWIFT_ENABLE_EXPERIMENTAL_{FEATURE}=YES`. Notice that `Concurrency` is an accepted language feature that should be enabled for Swift 5.5+. Test Swift: From c0315777d629cdbe409a005edb7f12e168cddf05 Mon Sep 17 00:00:00 2001 From: YR Chen Date: Mon, 28 Feb 2022 12:24:22 +0800 Subject: [PATCH 11/25] [WindowsBuild] Clearify for whom don't really need to build Swift --- docs/WindowsBuild.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index 1ce06e81cbee3..b0ae9454a6fb1 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -136,12 +136,14 @@ cmake --build S:\b\1 --target install The following guide will get you through the building process of a complete Swift debug toolchain. -Before you kick off the process, unset `SDKROOT` if you've already installed Swift: +Before you kick off the process, unset `%SDKROOT%` if you've already installed Swift: ```cmd set SDKROOT= ``` +If you want to use an existing Swift toolchain to build the core libraries and other parts of the toolchain, make sure you have it in `%Path%`, and strip `S:/b/1/bin/` from every `-D CMAKE_{language}_COMPILER=S:/b/1/bin/{compiler}` option. + ### Swift compiler ```cmd From 652ec206b09702298e9ef3d9851cf9b0054889a7 Mon Sep 17 00:00:00 2001 From: YR Chen Date: Mon, 28 Feb 2022 12:43:20 +0800 Subject: [PATCH 12/25] [WindowsBuild] SwiftSystem and TSC --- docs/WindowsBuild.md | 46 ++++++++++++++++++++++++++++++++++++-------- 1 file changed, 38 insertions(+), 8 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index b0ae9454a6fb1..c78b83ac3390a 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -350,23 +350,53 @@ Test Foundation: cmake --build S:\b\4 --target test ``` -### TSC +### SwiftSystem (without tests) ```cmd -cmake -B S:\b\5 ^ +cmake -B S:\b\6 ^ -D BUILD_SHARED_LIBS=YES ^ -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ - -D CMAKE_INSTALL_PREFIX=C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr ^ + -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ + -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ + -D CMAKE_MT=mt ^ + -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ - -D dispatch_DIR=S:\b\2\cmake\modules ^ - -D Foundation_DIR=S:\b\3\cmake\modules ^ - -D SQLite3_INCLUDE_DIR=S:\Library\sqlite-3.28.0\usr\include ^ - -D SQLite3_LIBRARY=S:\Library\sqlite-3.28.0\usr\lib\SQLite3.lib ^ + + -G Ninja ^ + -S S:\swift-system + +cmake --build S:\b\6 +``` + +### TSC (without tests) + +```cmd +cmake -B S:\b\7 ^ + -D BUILD_SHARED_LIBS=YES ^ + -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ + + -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ + -D CMAKE_MT=mt ^ + -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ + + -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ + -D dispatch_DIR=S:\b\3\cmake\modules ^ + -D Foundation_DIR=S:\b\4\cmake\modules ^ + -D SwiftSystem_DIR=S:\b\6\cmake\modules ^ + + -D CMAKE_TOOLCHAIN_FILE=S:\vcpkg\scripts\buildsystems\vcpkg.cmake ^ + -G Ninja ^ -S S:\swift-tools-support-core -ninja -C S:\b\5 +cmake --build S:\b\7 ``` ### llbuild From 26ed13b07f8af8fe785bbe242de24c187e1bbbeb Mon Sep 17 00:00:00 2001 From: YR Chen Date: Mon, 28 Feb 2022 17:31:19 +0800 Subject: [PATCH 13/25] [WindowsBuild] Guide up to Swift Driver --- docs/WindowsBuild.md | 213 ++++++++++++++++++++++++++++++++----------- 1 file changed, 160 insertions(+), 53 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index c78b83ac3390a..b3ed2219a2ddc 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -350,38 +350,55 @@ Test Foundation: cmake --build S:\b\4 --target test ``` -### SwiftSystem (without tests) +### LLBuild ```cmd cmake -B S:\b\6 ^ - -D BUILD_SHARED_LIBS=YES ^ -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ - -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ - -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ + -D CMAKE_CXX_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_CXX_FLAGS="/GS- /Oy /Gw /Gy -Xclang -fno-split-cold-code" ^ -D CMAKE_MT=mt ^ -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D LLBUILD_SUPPORT_BINDINGS=Swift ^ -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ + -D dispatch_DIR=S:\b\3\cmake\modules ^ + -D Foundation_DIR=S:\b\4\cmake\modules ^ + + -D CMAKE_TOOLCHAIN_FILE=S:\vcpkg\scripts\buildsystems\vcpkg.cmake ^ + + -D LIT_EXECUTABLE=S:\llvm-project\llvm\utils\lit\lit.py ^ + -D FILECHECK_EXECUTABLE=S:\b\1\bin\FileCheck.exe ^ -G Ninja ^ - -S S:\swift-system + -S S:\llbuild cmake --build S:\b\6 ``` -### TSC (without tests) +Test LLBuild: ```cmd -cmake -B S:\b\7 ^ +cmake --build S:\b\6 --target test +``` + +### Toolchain dependencies + +We're building the following libraries with `Release` preset and without tests here because they're independent packages that are directly or indirectly depended by Swift Driver, SwiftPM or SourceKit-LSP. For developing these libraries, use SwiftPM instead. + +#### Yams (used by Swift Driver) + +```cmd +cmake -B S:\b\7\Yams ^ -D BUILD_SHARED_LIBS=YES ^ - -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ - -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ + -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy /DYAML_DECLARE_EXPORT /DWIN32" ^ -D CMAKE_MT=mt ^ -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ @@ -389,99 +406,189 @@ cmake -B S:\b\7 ^ -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ -D dispatch_DIR=S:\b\3\cmake\modules ^ -D Foundation_DIR=S:\b\4\cmake\modules ^ - -D SwiftSystem_DIR=S:\b\6\cmake\modules ^ - -D CMAKE_TOOLCHAIN_FILE=S:\vcpkg\scripts\buildsystems\vcpkg.cmake ^ + -D BUILD_TESTING=NO ^ -G Ninja ^ - -S S:\swift-tools-support-core + -S S:\yams -cmake --build S:\b\7 +cmake --build S:\b\7\Yams ``` -### llbuild +#### Argument Parser (used by Swift Driver and SwiftPM) ```cmd -cmake -B S:\b\6 ^ +cmake -B S:\b\7\ArgumentParser ^ -D BUILD_SHARED_LIBS=YES ^ - -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ - -D CMAKE_INSTALL_PREFIX=C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr ^ - -D CMAKE_CXX_COMPILER=S:/b/1/bin/clang-cl.exe ^ - -D CMAKE_CXX_FLAGS="-Xclang -fno-split-cold-code" ^ + -D CMAKE_BUILD_TYPE=Release ^ + -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ + -D CMAKE_MT=mt ^ + -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ - -D LLBUILD_SUPPORT_BINDINGS=Swift ^ - -D dispatch_DIR=S:\b\2\cmake\modules ^ - -D Foundation_DIR=S:\b\3\cmake\modules ^ - -D SQLite3_INCLUDE_DIR=S:\Library\sqlite-3.28.0\usr\include ^ - -D SQLite3_LIBRARY=S:\Library\sqlite-3.28.0\usr\lib\sqlite3.lib ^ + -D dispatch_DIR=S:\b\3\cmake\modules ^ + -D Foundation_DIR=S:\b\4\cmake\modules ^ + + -D BUILD_EXAMPLES=NO ^ + -D BUILD_TESTING=NO ^ + -G Ninja ^ - -S S:\swift-llbuild + -S S:\swift-argument-parser -ninja -C S:\b\6 +cmake --build S:\b\7\ArgumentParser ``` -Add llbuild to your path: +#### Swift System (used by Swift Driver and SwiftPM) ```cmd -path S:\b\6\bin;%PATH% +cmake -B S:\b\7\SwiftSystem ^ + -D BUILD_SHARED_LIBS=YES ^ + -D CMAKE_BUILD_TYPE=Release ^ + -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ + + -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ + -D CMAKE_MT=mt ^ + -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ + + -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ + + -G Ninja ^ + -S S:\swift-system + +cmake --build S:\b\7\SwiftSystem ``` -### Yams +#### Swift Crypto (used by SwiftPM) ```cmd -cmake -B S:\b\7 ^ +cmake -B S:\b\7\SwiftCrypto ^ -D BUILD_SHARED_LIBS=YES ^ -D CMAKE_BUILD_TYPE=Release ^ - -D CMAKE_INSTALL_PREFIX=C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr ^ + -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ + + -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ -D CMAKE_MT=mt ^ + -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ - -D dispatch_DIR=S:\b\2\cmake\modules ^ - -D Foundation_DIR=S:\b\3\cmake\modules ^ - -D XCTest_DIR=S:\b\4\cmake\modules ^ + -D dispatch_DIR=S:\b\3\cmake\modules ^ + -D Foundation_DIR=S:\b\4\cmake\modules ^ + -G Ninja ^ - -S S:\Yams + -S S:\swift-crypto -ninja -C S:\b\7 +cmake --build S:\b\7\SwiftCrypto ``` -### ArgumentParser +#### Swift Collections (used by SwiftPM) ```cmd -cmake -B S:\b\8 ^ +cmake -B S:\b\7\SwiftCollections ^ -D BUILD_SHARED_LIBS=YES ^ -D CMAKE_BUILD_TYPE=Release ^ - -D CMAKE_INSTALL_PREFIX=C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr ^ + -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ + + -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ -D CMAKE_MT=mt ^ + -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ - -D dispatch_DIR=S:\b\2\cmake\modules ^ - -D Foundation_DIR=S:\b\3\cmake\modules ^ - -D XCTest_DIR=S:\b\4\cmake\modules ^ + + -D BUILD_TESTING=NO ^ + -G Ninja ^ - -S S:\swift-argument-parser + -S S:\swift-collections -ninja -C S:\b\8 +cmake --build S:\b\7\SwiftCollections ``` -### SwiftDriver +#### IndexStoreDB (used by SourceKit-LSP) ```cmd -cmake -B S:\b\9 ^ +cmake -B S:\b\7\IndexStoreDB ^ -D BUILD_SHARED_LIBS=YES ^ -D CMAKE_BUILD_TYPE=Release ^ - -D CMAKE_INSTALL_PREFIX=C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr ^ + -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ + + -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ + -D CMAKE_CXX_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_CXX_FLAGS="/GS- /Oy /Gw /Gy" ^ -D CMAKE_MT=mt ^ + -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ - -D dispatch_DIR=S:\b\2\cmake\modules ^ - -D Foundation_DIR=S:\b\3\cmake\modules ^ - -D TSC_DIR=S:\b\5\cmake\modules ^ + -D dispatch_DIR=S:\b\3\cmake\modules ^ + -D Foundation_DIR=S:\b\4\cmake\modules ^ + + -D BUILD_TESTING=NO ^ + + -G Ninja ^ + -S S:\indexstore-db + +cmake --build S:\b\7\IndexStoreDB +``` + +### TSC (without tests) + +```cmd +cmake -B S:\b\8 ^ + -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ + + -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ + -D CMAKE_MT=mt ^ + -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ + + -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ + -D dispatch_DIR=S:\b\3\cmake\modules ^ + -D Foundation_DIR=S:\b\4\cmake\modules ^ + -D SwiftSystem_DIR=S:\b\7\SwiftSystem\cmake\modules ^ + + -D CMAKE_TOOLCHAIN_FILE=S:\vcpkg\scripts\buildsystems\vcpkg.cmake ^ + + -G Ninja ^ + -S S:\swift-tools-support-core + +cmake --build S:\b\8 +``` + +### Swift Driver (without tests) + +```cmd +cmake -B S:\b\9 ^ + -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ + + -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ + -D CMAKE_MT=mt ^ + -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ + + -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ + -D dispatch_DIR=S:\b\3\cmake\modules ^ + -D Foundation_DIR=S:\b\4\cmake\modules ^ -D LLBuild_DIR=S:\b\6\cmake\modules ^ - -D Yams_DIR=S:\b\7\cmake\modules ^ - -D ArgumentParser_DIR=S:\b\8\cmake\modules ^ + -D Yams_DIR=S:\b\7\Yams\cmake\modules ^ + -D SwiftSystem_DIR=S:\b\7\SwiftSystem\cmake\modules ^ + -D ArgumentParser_DIR=S:\b\7\ArgumentParser\cmake\modules ^ + -D TSC_DIR=S:\b\8\cmake\modules ^ + -G Ninja ^ -S S:\swift-driver -ninja -C S:\b\9 +cmake --build S:\b\9 ``` ### SwiftPM From d267999ecb0672b995eedfd1aadcb785166c2860 Mon Sep 17 00:00:00 2001 From: YR Chen Date: Mon, 28 Feb 2022 17:59:50 +0800 Subject: [PATCH 14/25] [WindowsBuild] Restructure preparation steps --- docs/WindowsBuild.md | 74 +++++++++++++++++++++----------------------- 1 file changed, 35 insertions(+), 39 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index b3ed2219a2ddc..32138af2a9c47 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -2,17 +2,19 @@ Visual Studio 2017 or newer is needed to build Swift on Windows, while VS2019 is recommended and currently used for CI. The free Community edition is sufficient to build Swift, and we're assuming host and target to be both x64. -The commands below (with the exception of installing Visual Studio) must be entered in the **x64 Native Tools Command Prompt for VS2019** (or VS2017, VS2022 depending on the Visual Studio that you are using) in the Start Menu. This sets environment variables to select the correct target platform. +The commands below (with the exception of installing Visual Studio) must be entered in the **x64 Native Tools Command Prompt for VS2019** (or VS2017, VS2022 depending on the Visual Studio that you are using) in the Start Menu. This sets environment variables to locate tools and resources for the correct target platform. > **NOTE:** This guide is intended for toolchain developers who wants to develop or build Swift on their own machine. For building and packaging a standard toolchain, please refer to [`build-windows-toolchain.bat`](../utils/build-windows-toolchain.bat). -## Install dependencies +## Enable Developer Mode -### Visual Studio +From **Settings** app, go to *Update & Security*. In the *For developers* tab, set *Install apps from any source, including loose files* to On. This is required to enable the creation of symbolic links. -An easy way to get most of the tools to build Swift is using the [Visual Studio installer](https://www.visualstudio.com/downloads/). This command installs all needed Visual Studio components as well as Python, Git, CMake and Ninja: +## Install build tools and SDK -``` +An easy way to get the tools to build Swift is using the [Visual Studio installer](https://www.visualstudio.com/downloads/). The following command installs all needed Visual Studio components as well as Python, Git, CMake and Ninja: + +```cmd curl.exe -sOL https://aka.ms/vs/16/release/vs_community.exe vs_community ^ --add Component.CPython3.x64 ^ @@ -28,13 +30,9 @@ del /q vs_community.exe > - replace `https://aka.ms/vs/16/release/vs_community.exe` with `https://aka.ms/vs/17/release/vs_community.exe` > - replace `Component.CPython3.x64` with `Component.CPython39.x64` -If you prefer you can install everything by hand, but make sure to include "Programming Languages|Visual C++" and "Windows and Web Development|Universal Windows App Development|Windows SDK" in your installation. The components listed above are required. +> **NOTE:** The following [link](https://docs.microsoft.com/visualstudio/install/workload-component-id-vs-build-tools?view=vs-2019) helps in finding the component name given its ID for Visual Studio 2019. For anyone using VS installer in GUI, please check it out. -The following [link](https://docs.microsoft.com/visualstudio/install/workload-component-id-vs-build-tools?view=vs-2019) helps in finding the component name given its ID for Visual Studio 2019. - -### Python - -The command above already installs Python 3. Alternatively, in the Visual Studio installation program, under *Individual Components*, install *Python 3 64 bits (3.x.x)*. +### Configure Python If you are building a debug version of Swift, you should also install the Python debug binaries. @@ -44,45 +42,43 @@ If you are building a debug version of Swift, you should also install the Python 4. Select *Download debug binaries (requires VS 2015 or later)* 5. Click *Install* -## Enable Developer Mode - -From the settings application, go to *Update & Security*. In the *For developers* tab, select *Developer Mode* for *Use Developer Features*. This is required to enable the creation of symbolic links. +### Set up `vcpkg` -## Clone the repositories - -> **NOTE:** This guide assumes your sources live at the root of `S:`. If your sources live elsewhere, you can create a substitution for this: -> -> subst S: +This guide uses `vcpkg` for pulling in external dependencies, including ICU, libcurl, libxml2, SQLite 3 and zlib. -First, clone `apple/swift` (this repository) with Git: +All you have to do is to clone and bootstrap `vcpkg`: ```cmd -git clone -c core.autocrlf=input -c core.symlinks=true https://github.com/apple/swift S:\swift +git clone https://github.com/microsoft/vcpkg S:\vcpkg +S:\vcpkg\bootstrap-vcpkg.bat ``` -You'll be able to clone and check out the rest of Swift source repositories with `update-checkout` tool: +> **NOTE:** By default, the dependencies will be downloaded and built on demand. Optionally, you can pre-build the dependencies to speed up configuration: +> +> S:\vcpkg\vcpkg install curl icu libxml2 sqlite3 zlib --triplet=x64-windows -```cmd -S:\swift\utils\update-checkout.cmd --clone -``` +## Clone the source repositories -## Set up `vcpkg` +This guide assumes your sources live at the root of `S:`. As long as drive letter `S:` is not assigned, you can create a substitution to map your source directory: -This guide uses `vcpkg` for pulling in external dependencies, including ICU, libcurl, libxml2, SQLite 3 and zlib. +```cmd +subst S: +``` -All you have to do is to clone and bootstrap `vcpkg`: +First, clone `apple/swift` (this repository) with Git: ```cmd -git clone https://github.com/microsoft/vcpkg S:\vcpkg -S:\vcpkg\bootstrap-vcpkg.bat +git clone -c core.autocrlf=input -c core.symlinks=true https://github.com/apple/swift S:\swift ``` -By default, the dependencies will be downloaded and built on demand. Optionally, you can prebuild the dependencies in advance: +You'll be able to clone and check out the rest of Swift source repositories with `update-checkout` tool: ```cmd -S:\vcpkg\vcpkg install curl icu libxml2 sqlite3 zlib --triplet=x64-windows +S:\swift\utils\update-checkout.cmd --clone --skip-repository swift ``` +> **NOTE:** You can use forked versions of Swift source repositories by manually placing them into `S:\` and appending their names to the `--skip-repository` option of `update-checkout`. Don't forget to clone them with `-c core.autocrlf=input -c core.symlinks=true` and the correct repository name. + ## Set up Visual Studio integration (re-run on Visual Studio upgrades) Set up the `ucrt`, `visualc`, and `WinSDK` modules by: @@ -113,9 +109,9 @@ cmake -B S:\b\1 ^ -C S:\swift\cmake\caches\Windows-x86_64.cmake ^ -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr ^ - -D CMAKE_C_COMPILER=cl ^ + -D CMAKE_C_COMPILER=cl.exe ^ -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy /source-charset:utf-8 /execution-charset:utf-8" ^ - -D CMAKE_CXX_COMPILER=cl ^ + -D CMAKE_CXX_COMPILER=cl.exe ^ -D CMAKE_CXX_FLAGS="/GS- /Oy /Gw /Gy /utf-8" ^ -D CMAKE_MT=mt ^ -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ @@ -132,7 +128,7 @@ cmake --build S:\b\1 cmake --build S:\b\1 --target install ``` -## Build a complete toolchain for development +## Build for development The following guide will get you through the building process of a complete Swift debug toolchain. @@ -152,9 +148,9 @@ cmake -B S:\b\1 ^ -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ - -D CMAKE_C_COMPILER=cl ^ + -D CMAKE_C_COMPILER=cl.exe ^ -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy /source-charset:utf-8 /execution-charset:utf-8" ^ - -D CMAKE_CXX_COMPILER=cl ^ + -D CMAKE_CXX_COMPILER=cl.exe ^ -D CMAKE_CXX_FLAGS="/GS- /Oy /Gw /Gy /utf-8" ^ -D CMAKE_MT=mt ^ -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ @@ -192,7 +188,7 @@ path %PATH%;%ProgramFiles%\Git\usr\bin cmake --build S:\b\1 --target check-swift ``` -### Swift Standard Library +### Swift runtime libraries ```cmd cmake -B S:\b\2 ^ @@ -225,7 +221,7 @@ cmake -B S:\b\2 ^ cmake --build S:\b\2 ``` -> **NOTE:** Swift Standard Library is also built along with the compiler. This step extracts it into a portable SDK where we will install other runtime libraries. +> **NOTE:** Swift runtime libraries are also built along with the compiler. This step extracts them into a portable SDK where we will install core libraries alongside. ### libdispatch From 55460134b49a3503937914f91e06987bd911ee62 Mon Sep 17 00:00:00 2001 From: YR Chen Date: Mon, 28 Feb 2022 18:56:09 +0800 Subject: [PATCH 15/25] [WindowsBuild] SwiftPM and SourceKit-LSP --- docs/WindowsBuild.md | 59 +++++++++++++++++++++++++++++++++++--------- 1 file changed, 47 insertions(+), 12 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index 32138af2a9c47..31b5b0abec7aa 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -516,7 +516,7 @@ cmake -B S:\b\7\IndexStoreDB ^ -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ -D CMAKE_CXX_COMPILER=S:/b/1/bin/clang-cl.exe ^ - -D CMAKE_CXX_FLAGS="/GS- /Oy /Gw /Gy" ^ + -D CMAKE_CXX_FLAGS="/GS- /Oy /Gw /Gy -Xclang -fno-split-cold-code" ^ -D CMAKE_MT=mt ^ -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ @@ -537,6 +537,7 @@ cmake --build S:\b\7\IndexStoreDB ```cmd cmake -B S:\b\8 ^ + -D BUILD_SHARED_LIBS=YES ^ -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ @@ -563,6 +564,7 @@ cmake --build S:\b\8 ```cmd cmake -B S:\b\9 ^ + -D BUILD_SHARED_LIBS=YES ^ -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ @@ -587,33 +589,66 @@ cmake -B S:\b\9 ^ cmake --build S:\b\9 ``` -### SwiftPM +### SwiftPM (without tests) ```cmd cmake -B S:\b\10 ^ -D BUILD_SHARED_LIBS=YES ^ - -D CMAKE_BUILD_TYPE=Release ^ + -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ + -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ - -D CMAKE_INSTALL_PREFIX=C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr ^ + -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ -D CMAKE_MT=mt ^ + -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ - -D dispatch_DIR=S:\b\2\cmake\modules ^ - -D Foundation_DIR=S:\b\3\cmake\modules ^ - -D TSC_DIR=S:\b\5\cmake\modules ^ + -D dispatch_DIR=S:\b\3\cmake\modules ^ + -D Foundation_DIR=S:\b\4\cmake\modules ^ -D LLBuild_DIR=S:\b\6\cmake\modules ^ - -D Yams_DIR=S:\b\7\cmake\modules ^ - -D ArgumentParser_DIR=S:\b\8\cmake\modules ^ + -D SwiftSystem_DIR=S:\b\7\SwiftSystem\cmake\modules ^ + -D ArgumentParser_DIR=S:\b\7\ArgumentParser\cmake\modules ^ + -D SwiftCrypto_DIR=S:\b\7\SwiftCrypto\cmake\modules ^ + -D SwiftCollections_DIR=S:\b\7\SwiftCollections\cmake\modules ^ + -D TSC_DIR=S:\b\8\cmake\modules ^ -D SwiftDriver_DIR=S:\b\9\cmake\modules ^ + -G Ninja ^ -S S:\swiftpm -ninja -C S:\b\10 +cmake --build S:\b\10 ``` -Indicate to SwiftPM where to find `PackageDescription` before installation: +### SourceKit-LSP (without tests) ```cmd -set SWIFTPM_PD_LIBS=S:\b\10\pm +cmake -B S:\b\11 ^ + -D BUILD_SHARED_LIBS=YES ^ + -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ + + -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ + -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy" ^ + -D CMAKE_MT=mt ^ + -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ + -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ + + -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ + -D dispatch_DIR=S:\b\3\cmake\modules ^ + -D Foundation_DIR=S:\b\4\cmake\modules ^ + -D LLBuild_DIR=S:\b\6\cmake\modules ^ + -D SwiftSystem_DIR=S:\b\7\SwiftSystem\cmake\modules ^ + -D ArgumentParser_DIR=S:\b\7\ArgumentParser\cmake\modules ^ + -D SwiftCollections_DIR=S:\b\7\SwiftCollections\cmake\modules ^ + -D IndexStoreDB_DIR=S:\b\7\IndexStoreDB\cmake\modules ^ + -D TSC_DIR=S:\b\8\cmake\modules ^ + -D SwiftPM_DIR=S:\b\10\cmake\modules ^ + + -G Ninja ^ + -S S:\sourcekit-lsp + +cmake --build S:\b\11 ``` ## Gather the toolchain and SDK From bbff38ef84682dc6260d177e8f86775f10942444 Mon Sep 17 00:00:00 2001 From: YR Chen Date: Mon, 28 Feb 2022 18:57:33 +0800 Subject: [PATCH 16/25] [WindowsBuild] Complete install guide --- docs/WindowsBuild.md | 33 +++++++++++++++++++++++++++++---- 1 file changed, 29 insertions(+), 4 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index 31b5b0abec7aa..d07e8a2c0dd2a 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -653,9 +653,9 @@ cmake --build S:\b\11 ## Gather the toolchain and SDK -If you want a toolchain for real-world testing or distribution, you can use CMake to perform the install step. +If you want a toolchain for real-world testing, you can use CMake to perform the install step. -### Swift compiler and standard library +### Swift compiler and standard libraries ```cmd cmake --build S:\b\1 --target install @@ -667,16 +667,41 @@ For testing, add the target to path: path S:\b\toolchain\usr\bin:%PATH% ``` -## Swift Windows SDK (with core libraries) +### Swift Windows SDK (with core libraries) ```cmd cmake --build S:\b\2 --target install cmake --build S:\b\3 --target install cmake --build S:\b\4 --target install +cmake --build S:\b\5 --target install ``` -For testing, set `SDKROOT`: +For testing, set `%SDKROOT%` and the default runtime: ```cmd set SDKROOT=S:\b\sdk +python -c "import plistlib; print(str(plistlib.dumps({ 'DefaultProperties': { 'DEFAULT_USE_RUNTIME': 'MD' } }), encoding='utf-8'))" > S:\b\sdk\SDKSettings.plist +``` + +### Swift toolchain with developer tools + +```cmd +cmake --build S:\b\1 --target install +cmake --build S:\b\6 --target install +cmake --build S:\b\7\Yams --target install +cmake --build S:\b\7\ArgumentParser --target install +cmake --build S:\b\7\SwiftSystem --target install +cmake --build S:\b\7\SwiftCollections --target install +cmake --build S:\b\7\IndexStoreDB --target install +cmake --build S:\b\8 --target install +cmake --build S:\b\9 --target install +cmake --build S:\b\10 --target install +cmake --build S:\b\11 --target install +``` + +To use Swift Driver in place of the old driver: + +```cmd +copy /Y S:\b\toolchain\usr\bin\swift-driver.exe S:\b\toolchain\usr\bin\swift.exe +copy /Y S:\b\toolchain\usr\bin\swift-driver.exe S:\b\toolchain\usr\bin\swiftc.exe ``` From 412e2e58e4ed6805018ff498de2a194729ba2d08 Mon Sep 17 00:00:00 2001 From: stevapple Date: Mon, 28 Feb 2022 23:05:00 +0800 Subject: [PATCH 17/25] [WindowsBuild] Graphical overview --- docs/WindowsBuild.md | 78 +++++++++++++++++++++++++++++++++++++------- 1 file changed, 66 insertions(+), 12 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index d07e8a2c0dd2a..1ae410861eea4 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -140,7 +140,61 @@ set SDKROOT= If you want to use an existing Swift toolchain to build the core libraries and other parts of the toolchain, make sure you have it in `%Path%`, and strip `S:/b/1/bin/` from every `-D CMAKE_{language}_COMPILER=S:/b/1/bin/{compiler}` option. -### Swift compiler +Here is a graphical overview of all build steps from this guide. + +``` + ┌──────────────────────────────────────┐ ┌────────────────┐ + │ │ │ │ + │ 1. LLVM + Swift compiler │ │ │ + │ │ │ Existing Swift │ + │ ┌────────────────────────────┐ │ or │ toolchain │ + │ │ 2. Swift runtime libraries │ │ │ │ + │ └────────────────────────────┘ │ │ │ + └──────────────────────────────────────┘ └────────────────┘ + │ │ + └─────────┬────────────────────────┘ + │ + ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ + ┌──────────────────────┐ │ + │ │ 3. libdispatch │ + └──────────────────────┘ │ + │ Core ┌──────────────────────┐ + ┌─ │ 4. Foundation │ │─┐ + │ │ libraries └──────────────────────┘ │ + │ ┌──────────────────────┐ │ │ + │ │ │ 5. XCTest │ │ + │ └──────────────────────┘ │ │ + │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ + │ │ + │ ┌──────────────────────────────────────┐ + │ │ │ + │ │ 7. Toolchain dependencies │ +┌────────────────────────────┐ │ │ +│ 6. LLBuild │ ├────┬───────────┬────────┬────────────┤ +└────────────────────────────┘ │Yams│SwiftSystem│ ··· │IndexStoreDB│ + │ └────┴───────────┴────────┴────────────┘ + │ │ + │ │ + │ ┌────────────────────────────┐ + │ │ 8. TSC │ + │ └────────────────────────────┘ + │ │ + └─────────────────────┬────────────────────┘ + │ + ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ + ┌──────────────────────┐ │ + │ │ 9. Swift Driver │ + └──────────────────────┘ │ + │ Developer ┌──────────────────────┐ + │ 10. SwiftPM │ │ + │ tools └──────────────────────┘ + ┌──────────────────────┐ │ + │ │ 11. SourceKit-LSP │ + └──────────────────────┘ │ + └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ +``` + +### 1. Swift compiler ```cmd cmake -B S:\b\1 ^ @@ -188,7 +242,7 @@ path %PATH%;%ProgramFiles%\Git\usr\bin cmake --build S:\b\1 --target check-swift ``` -### Swift runtime libraries +### 2. Swift runtime libraries ```cmd cmake -B S:\b\2 ^ @@ -223,7 +277,7 @@ cmake --build S:\b\2 > **NOTE:** Swift runtime libraries are also built along with the compiler. This step extracts them into a portable SDK where we will install core libraries alongside. -### libdispatch +### 3. libdispatch ```cmd cmake -B S:\b\3 ^ @@ -255,7 +309,7 @@ Test libdispatch: cmake --build S:\b\3 --target test ``` -### Foundation (without tests) +### 4. Foundation (without tests) ```cmd cmake -B S:\b\4 ^ @@ -279,7 +333,7 @@ cmake -B S:\b\4 ^ cmake --build S:\b\4 ``` -### XCTest +### 5. XCTest ```cmd cmake -B S:\b\5 ^ @@ -313,7 +367,7 @@ path %PATH%;%ProgramFiles%\Git\usr\bin cmake --build S:\b\5 --target check-xctest ``` -### Foundation (with tests) +### 4.1 Foundation (with tests) ```cmd cmake -B S:\b\4 ^ @@ -346,7 +400,7 @@ Test Foundation: cmake --build S:\b\4 --target test ``` -### LLBuild +### 6. LLBuild ```cmd cmake -B S:\b\6 ^ @@ -381,7 +435,7 @@ Test LLBuild: cmake --build S:\b\6 --target test ``` -### Toolchain dependencies +### 7. Toolchain dependencies We're building the following libraries with `Release` preset and without tests here because they're independent packages that are directly or indirectly depended by Swift Driver, SwiftPM or SourceKit-LSP. For developing these libraries, use SwiftPM instead. @@ -533,7 +587,7 @@ cmake -B S:\b\7\IndexStoreDB ^ cmake --build S:\b\7\IndexStoreDB ``` -### TSC (without tests) +### 8. TSC (without tests) ```cmd cmake -B S:\b\8 ^ @@ -560,7 +614,7 @@ cmake -B S:\b\8 ^ cmake --build S:\b\8 ``` -### Swift Driver (without tests) +### 9. Swift Driver (without tests) ```cmd cmake -B S:\b\9 ^ @@ -589,7 +643,7 @@ cmake -B S:\b\9 ^ cmake --build S:\b\9 ``` -### SwiftPM (without tests) +### 10. SwiftPM (without tests) ```cmd cmake -B S:\b\10 ^ @@ -620,7 +674,7 @@ cmake -B S:\b\10 ^ cmake --build S:\b\10 ``` -### SourceKit-LSP (without tests) +### 11. SourceKit-LSP (without tests) ```cmd cmake -B S:\b\11 ^ From 69d0547efbedb5930e4612c580fdc0ec578a2667 Mon Sep 17 00:00:00 2001 From: stevapple Date: Mon, 28 Feb 2022 23:14:14 +0800 Subject: [PATCH 18/25] [WindowsBuild] Minor fixes --- docs/WindowsBuild.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index 1ae410861eea4..317698a7ed773 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -260,6 +260,8 @@ cmake -B S:\b\2 ^ -D LLVM_DIR=S:\b\1\lib\cmake\llvm ^ -D SWIFT_NATIVE_SWIFT_TOOLS_PATH=S:\b\1\bin ^ + + -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ -D SWIFT_PATH_TO_LIBDISPATCH_SOURCE=S:\swift-corelibs-libdispatch ^ -D SWIFT_ENABLE_EXPERIMENTAL_CONCURRENCY=YES ^ @@ -745,6 +747,7 @@ cmake --build S:\b\6 --target install cmake --build S:\b\7\Yams --target install cmake --build S:\b\7\ArgumentParser --target install cmake --build S:\b\7\SwiftSystem --target install +cmake --build S:\b\7\SwiftCrypto --target install cmake --build S:\b\7\SwiftCollections --target install cmake --build S:\b\7\IndexStoreDB --target install cmake --build S:\b\8 --target install @@ -759,3 +762,10 @@ To use Swift Driver in place of the old driver: copy /Y S:\b\toolchain\usr\bin\swift-driver.exe S:\b\toolchain\usr\bin\swift.exe copy /Y S:\b\toolchain\usr\bin\swift-driver.exe S:\b\toolchain\usr\bin\swiftc.exe ``` + +Add the built toolchain to `%Path%` to use directly: + +```cmd +path S:\b\toolchain\usr\bin;%Path% +swift --version +``` From 54d3391890633e89561493a319d7bf861f16002b Mon Sep 17 00:00:00 2001 From: stevapple Date: Mon, 7 Mar 2022 05:28:27 +0800 Subject: [PATCH 19/25] [WindowsBuild] Deprecate Python 3.7 --- docs/WindowsBuild.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index 317698a7ed773..266df5f632e72 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -17,7 +17,7 @@ An easy way to get the tools to build Swift is using the [Visual Studio installe ```cmd curl.exe -sOL https://aka.ms/vs/16/release/vs_community.exe vs_community ^ - --add Component.CPython3.x64 ^ + --add Component.CPython39.x64 ^ --add Microsoft.VisualStudio.Component.Git ^ --add Microsoft.VisualStudio.Component.VC.ATL ^ --add Microsoft.VisualStudio.Component.VC.CMake.Project ^ @@ -26,9 +26,7 @@ vs_community ^ del /q vs_community.exe ``` -> **NOTE:** For anyone who wants to use Visual Studio 2022 instead: -> - replace `https://aka.ms/vs/16/release/vs_community.exe` with `https://aka.ms/vs/17/release/vs_community.exe` -> - replace `Component.CPython3.x64` with `Component.CPython39.x64` +> **NOTE:** For anyone who wants to use Visual Studio 2022, you can simply replace version number `16` with `17` in the installer URL. > **NOTE:** The following [link](https://docs.microsoft.com/visualstudio/install/workload-component-id-vs-build-tools?view=vs-2019) helps in finding the component name given its ID for Visual Studio 2019. For anyone using VS installer in GUI, please check it out. @@ -37,9 +35,9 @@ del /q vs_community.exe If you are building a debug version of Swift, you should also install the Python debug binaries. 1. In the Windows settings, go to *Add and Remove Programs* -2. Select the *Python 3.x.x (64-bit)* entry +2. Select the *Python 3.9.x (64-bit)* entry 3. Click *Modify*, then *Yes*, then *Modify* again and then *Next* -4. Select *Download debug binaries (requires VS 2015 or later)* +4. Select *Download debug binaries (requires VS 2017 or later)* 5. Click *Install* ### Set up `vcpkg` From 3003b0890509001b24525c0a2522177327958a45 Mon Sep 17 00:00:00 2001 From: YR Chen Date: Mon, 14 Mar 2022 15:07:26 +0800 Subject: [PATCH 20/25] [WindowsBuild] Update installation guide --- docs/WindowsBuild.md | 64 +++++++++++++++++++++++++++++++++----------- 1 file changed, 48 insertions(+), 16 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index 266df5f632e72..e2b0437955ef4 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -705,22 +705,24 @@ cmake -B S:\b\11 ^ cmake --build S:\b\11 ``` -## Gather the toolchain and SDK +## Use the toolchain and SDK -If you want a toolchain for real-world testing, you can use CMake to perform the install step. +If you want a toolchain for real-world testing, you can use CMake to perform the install step. However, the built toolchain is not expected to be distributed, mainly because there're plenty of unnecessary or local files that need to be stripped. -### Swift compiler and standard libraries +### Swift compiler ```cmd cmake --build S:\b\1 --target install ``` -For testing, add the target to path: +For testing, add the toolchain to path: ```cmd -path S:\b\toolchain\usr\bin:%PATH% +path S:\b\toolchain\usr\bin;%PATH% ``` +> **NOTE**: The built compiler cannot actually compile Swift files without a compatible SDK. You may need to specify `-sdk ` explicitly. + ### Swift Windows SDK (with core libraries) ```cmd @@ -730,17 +732,51 @@ cmake --build S:\b\4 --target install cmake --build S:\b\5 --target install ``` -For testing, set `%SDKROOT%` and the default runtime: +Usually, you also need to bundle external libraries required by `Foundation`: + +```cmd +copy /Y S:\b\4\bin\*.dll S:\b\sdk\usr\bin +``` + +For testing, set `%SDKROOT%` and add runtime libraries to path: ```cmd set SDKROOT=S:\b\sdk -python -c "import plistlib; print(str(plistlib.dumps({ 'DefaultProperties': { 'DEFAULT_USE_RUNTIME': 'MD' } }), encoding='utf-8'))" > S:\b\sdk\SDKSettings.plist +path S:\b\sdk\usr\bin;%PATH% ``` -### Swift toolchain with developer tools +#### Fix SDK layout + +You may notice that the SDK is not functioning correctly. This is caused by the mismatch of expected SDK layout and the build output. Before these issues get addressed, we may need some extra workaround. + +1. Compiler expects Swift libraries to live in `\usr\lib\swift\windows\x86_64`, but the build output is `\usr\lib\swift\windows`. This is likely to trigger `unable to load standard library for target 'x86_64-unknown-windows-msvc'` error. Fix by: + +```cmd +for /D %m in (S:\b\sdk\usr\lib\swift\windows\*.swiftmodule) do ^ +move /Y %m S:\b\sdk\usr\lib\swift\windows\x86_64 +``` + +> **NOTE(stevapple)**: The compiler convention shall be wrong here — Swift libraries can be bundled with multiple architectures, so there's no need to place them in arch-specific directory. + +2. Compiler expects underlying modules to be imported from `\usr\include`, but the build output is `\usr\lib\swift`. This is likely to trigger `cannot load underlying module for ''` error. Fix by: + +```cmd +move /Y S:\b\sdk\usr\lib\swift\Block S:\b\sdk\usr\include +move /Y S:\b\sdk\usr\lib\swift\dispatch S:\b\sdk\usr\include +move /Y S:\b\sdk\usr\lib\swift\os S:\b\sdk\usr\include +``` + +3. Compiler expects Swift runtime import libraries to live in `\usr\lib\swift\windows\x86_64`, but the build output is `\usr\lib\swift\windows`. This is likely to trigger `fatal error LNK1104: cannot open file '.lib'` error. Fix by: + +```cmd +move /Y S:\b\sdk\usr\lib\swift\windows\*.lib S:\b\sdk\usr\lib\swift\windows\x86_64 +``` + +> **NOTE(stevapple)**: The build script shall be wrong here — Swift shared libraries are closely tied to target architecture, so we should place them in arch-specific directory if Universal binary is not available. + +### Swift developer tools ```cmd -cmake --build S:\b\1 --target install cmake --build S:\b\6 --target install cmake --build S:\b\7\Yams --target install cmake --build S:\b\7\ArgumentParser --target install @@ -752,18 +788,14 @@ cmake --build S:\b\8 --target install cmake --build S:\b\9 --target install cmake --build S:\b\10 --target install cmake --build S:\b\11 --target install +copy /Y S:\b\8\bin\sqlite3.dll S:\b\toolchain\usr\bin ``` -To use Swift Driver in place of the old driver: +To use Swift Driver in place of the old driver (default since Swift 5.7): ```cmd copy /Y S:\b\toolchain\usr\bin\swift-driver.exe S:\b\toolchain\usr\bin\swift.exe copy /Y S:\b\toolchain\usr\bin\swift-driver.exe S:\b\toolchain\usr\bin\swiftc.exe ``` -Add the built toolchain to `%Path%` to use directly: - -```cmd -path S:\b\toolchain\usr\bin;%Path% -swift --version -``` +You should be able to compile Swift packages without any additional steps. From cab43f47b68b1a9c5b6c25eda94ed9e346486418 Mon Sep 17 00:00:00 2001 From: YR Chen Date: Sun, 3 Apr 2022 02:12:25 +0800 Subject: [PATCH 21/25] [WindowsBuild] Update VS version --- docs/WindowsBuild.md | 24 +++++++----------------- 1 file changed, 7 insertions(+), 17 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index e2b0437955ef4..fabcbf312345c 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -1,8 +1,8 @@ # Building Swift on Windows -Visual Studio 2017 or newer is needed to build Swift on Windows, while VS2019 is recommended and currently used for CI. The free Community edition is sufficient to build Swift, and we're assuming host and target to be both x64. +Visual Studio 2019 or newer is needed to build Swift on Windows, while VS2022 is recommended and currently used for CI. The free Community edition is sufficient to build Swift, and we're assuming host and target to be both x64. -The commands below (with the exception of installing Visual Studio) must be entered in the **x64 Native Tools Command Prompt for VS2019** (or VS2017, VS2022 depending on the Visual Studio that you are using) in the Start Menu. This sets environment variables to locate tools and resources for the correct target platform. +The commands below (with the exception of installing Visual Studio) must be entered in the **x64 Native Tools Command Prompt for VS2022** (or VS2019 depending on the Visual Studio that you are using) in the Start Menu. This sets environment variables to locate tools and resources for the correct target platform. > **NOTE:** This guide is intended for toolchain developers who wants to develop or build Swift on their own machine. For building and packaging a standard toolchain, please refer to [`build-windows-toolchain.bat`](../utils/build-windows-toolchain.bat). @@ -15,30 +15,20 @@ From **Settings** app, go to *Update & Security*. In the *For developers* tab, An easy way to get the tools to build Swift is using the [Visual Studio installer](https://www.visualstudio.com/downloads/). The following command installs all needed Visual Studio components as well as Python, Git, CMake and Ninja: ```cmd -curl.exe -sOL https://aka.ms/vs/16/release/vs_community.exe +curl.exe -sOL https://aka.ms/vs/17/release/vs_community.exe vs_community ^ --add Component.CPython39.x64 ^ --add Microsoft.VisualStudio.Component.Git ^ --add Microsoft.VisualStudio.Component.VC.ATL ^ --add Microsoft.VisualStudio.Component.VC.CMake.Project ^ --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 ^ - --add Microsoft.VisualStudio.Component.Windows10SDK.19041 + --add Microsoft.VisualStudio.Component.Windows11SDK.22000 del /q vs_community.exe ``` -> **NOTE:** For anyone who wants to use Visual Studio 2022, you can simply replace version number `16` with `17` in the installer URL. +> **NOTE:** For anyone who wants to use Visual Studio 2019, you can simply replace version number `17` with `16` in the installer URL. -> **NOTE:** The following [link](https://docs.microsoft.com/visualstudio/install/workload-component-id-vs-build-tools?view=vs-2019) helps in finding the component name given its ID for Visual Studio 2019. For anyone using VS installer in GUI, please check it out. - -### Configure Python - -If you are building a debug version of Swift, you should also install the Python debug binaries. - -1. In the Windows settings, go to *Add and Remove Programs* -2. Select the *Python 3.9.x (64-bit)* entry -3. Click *Modify*, then *Yes*, then *Modify* again and then *Next* -4. Select *Download debug binaries (requires VS 2017 or later)* -5. Click *Install* +> **NOTE:** The following [link](https://docs.microsoft.com/visualstudio/install/workload-component-id-vs-build-tools?view=vs-2022) helps in finding the component name given its ID for Visual Studio 2022. For anyone using VS installer in GUI, please check it out. ### Set up `vcpkg` @@ -100,7 +90,7 @@ mklink "%VCToolsInstallDir%\include\visualc.apinotes" S:\swift\stdlib\public\Pla ## Build a minimal Swift toolchain -A minimal Swift toolchain comes with neither optional features nor SwiftPM stuffs, and is useful for playing with the compiler or language itself. +A minimal Swift toolchain does not include optional features or SwiftPM stuffs, and may be useful for playing with the compiler or language itself. ```cmd cmake -B S:\b\1 ^ From f52b77e615d35f149d54d8f6bfbf7596c8e2e1bc Mon Sep 17 00:00:00 2001 From: YR Chen Date: Sun, 3 Apr 2022 02:20:18 +0800 Subject: [PATCH 22/25] [WindowsBuild] Fix debug info --- docs/WindowsBuild.md | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index fabcbf312345c..2b9e71e8ed34e 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -199,6 +199,7 @@ cmake -B S:\b\1 ^ -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ -D LLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-windows-msvc ^ + -D LLVM_ENABLE_PDB=YES ^ -D LLVM_EXTERNAL_CMARK_SOURCE_DIR=S:\cmark ^ -D LLVM_EXTERNAL_SWIFT_SOURCE_DIR=S:\swift ^ @@ -217,11 +218,11 @@ cmake -B S:\b\1 ^ cmake --build S:\b\1 ``` -> **NOTE:** If you want to profile the Swift compiler, you may need to link it with debug information. You can enable this by specifying `-D LLVM_ENABLE_PDB=YES`. +> **NOTE:** Flag `-D LLVM_ENABLE_PDB=YES` enables LLVM to emit PDB files, which is useful for debugging on Windows. However, linking with debug information is very memory-intensive and may drastically slow down the linking process. A single link job is possible to consume upwards of 10 GiB of RAM. You can append `-D LLVM_PARALLEL_LINK_JOBS=N` and `-D SWIFT_PARALLEL_LINK_JOBS=N` to reduce the number of parallel link operations to `N` which should help reduce the memory pressure. > -> Linking with debug information is very memory-intensive and may drastically slow down the linking process. A single link job is possible to consume upwards of 10 GiB of RAM. You can append `-D LLVM_PARALLEL_LINK_JOBS=N` to reduce the number of parallel link operations to `N` which should help reduce the memory pressure. +> For host with less than 32 GiB of RAM, you may need to disable PDB support by stripping this option, to avoid running into OOM. -> **NOTE:** By default, we enables all the experimental features in Swift by `-D SWIFT_ENABLE_EXPERIMENTAL_{FEATURE}=YES`. Notice that `Concurrency` is an accepted language feature that should be enabled for Swift 5.5+. +> **NOTE:** By default, we enables all the experimental features in Swift by `-D SWIFT_ENABLE_EXPERIMENTAL_{FEATURE}=YES`. Notice that `Concurrency` is an accepted language feature that is supposed be enabled for Swift 5.5+. Test Swift: @@ -235,7 +236,7 @@ cmake --build S:\b\1 --target check-swift ```cmd cmake -B S:\b\2 ^ -C S:\swift\cmake\caches\Runtime-Windows-x86_64.cmake ^ - -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\sdk\usr ^ -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ @@ -271,7 +272,7 @@ cmake --build S:\b\2 ```cmd cmake -B S:\b\3 ^ - -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\sdk\usr ^ -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ @@ -303,7 +304,7 @@ cmake --build S:\b\3 --target test ```cmd cmake -B S:\b\4 ^ - -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\sdk\usr ^ -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ @@ -327,7 +328,7 @@ cmake --build S:\b\4 ```cmd cmake -B S:\b\5 ^ - -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\sdk\usr ^ -D CMAKE_MT=mt ^ @@ -361,7 +362,7 @@ cmake --build S:\b\5 --target check-xctest ```cmd cmake -B S:\b\4 ^ - -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\sdk\usr ^ -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ @@ -394,7 +395,7 @@ cmake --build S:\b\4 --target test ```cmd cmake -B S:\b\6 ^ - -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ -D CMAKE_CXX_COMPILER=S:/b/1/bin/clang-cl.exe ^ @@ -582,7 +583,7 @@ cmake --build S:\b\7\IndexStoreDB ```cmd cmake -B S:\b\8 ^ -D BUILD_SHARED_LIBS=YES ^ - -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ @@ -609,7 +610,7 @@ cmake --build S:\b\8 ```cmd cmake -B S:\b\9 ^ -D BUILD_SHARED_LIBS=YES ^ - -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ @@ -638,7 +639,7 @@ cmake --build S:\b\9 ```cmd cmake -B S:\b\10 ^ -D BUILD_SHARED_LIBS=YES ^ - -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ @@ -669,7 +670,7 @@ cmake --build S:\b\10 ```cmd cmake -B S:\b\11 ^ -D BUILD_SHARED_LIBS=YES ^ - -D CMAKE_BUILD_TYPE=RelWithDebInfo ^ + -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ -D CMAKE_C_COMPILER=S:/b/1/bin/clang-cl.exe ^ From f2c484c84488c99953a14d6219d1d5c2383e0f7a Mon Sep 17 00:00:00 2001 From: YR Chen Date: Sun, 3 Apr 2022 16:44:03 +0800 Subject: [PATCH 23/25] [WindowsBuild] Rearrange build steps --- docs/WindowsBuild.md | 126 +++++++++++++++++++++---------------------- 1 file changed, 60 insertions(+), 66 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index 2b9e71e8ed34e..1851cf212a84a 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -391,10 +391,14 @@ Test Foundation: cmake --build S:\b\4 --target test ``` -### 6. LLBuild +### 6. Tool dependencies + +> **NOTE:** We're building the following libraries without tests here because they're independent packages that are directly or indirectly depended by Swift Driver, SwiftPM or SourceKit-LSP. For developing these libraries, use SwiftPM instead. + +#### LLBuild (Used by Swift Driver, SwiftPM and SourceKit-LSP) ```cmd -cmake -B S:\b\6 ^ +cmake -B S:\b\6\LLBuild ^ -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ @@ -417,23 +421,13 @@ cmake -B S:\b\6 ^ -G Ninja ^ -S S:\llbuild -cmake --build S:\b\6 -``` - -Test LLBuild: - -```cmd -cmake --build S:\b\6 --target test +cmake --build S:\b\6\LLBuild ``` -### 7. Toolchain dependencies - -We're building the following libraries with `Release` preset and without tests here because they're independent packages that are directly or indirectly depended by Swift Driver, SwiftPM or SourceKit-LSP. For developing these libraries, use SwiftPM instead. - #### Yams (used by Swift Driver) ```cmd -cmake -B S:\b\7\Yams ^ +cmake -B S:\b\6\Yams ^ -D BUILD_SHARED_LIBS=YES ^ -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ @@ -453,13 +447,13 @@ cmake -B S:\b\7\Yams ^ -G Ninja ^ -S S:\yams -cmake --build S:\b\7\Yams +cmake --build S:\b\6\Yams ``` #### Argument Parser (used by Swift Driver and SwiftPM) ```cmd -cmake -B S:\b\7\ArgumentParser ^ +cmake -B S:\b\6\ArgumentParser ^ -D BUILD_SHARED_LIBS=YES ^ -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ @@ -477,13 +471,13 @@ cmake -B S:\b\7\ArgumentParser ^ -G Ninja ^ -S S:\swift-argument-parser -cmake --build S:\b\7\ArgumentParser +cmake --build S:\b\6\ArgumentParser ``` #### Swift System (used by Swift Driver and SwiftPM) ```cmd -cmake -B S:\b\7\SwiftSystem ^ +cmake -B S:\b\6\SwiftSystem ^ -D BUILD_SHARED_LIBS=YES ^ -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ @@ -499,13 +493,13 @@ cmake -B S:\b\7\SwiftSystem ^ -G Ninja ^ -S S:\swift-system -cmake --build S:\b\7\SwiftSystem +cmake --build S:\b\6\SwiftSystem ``` #### Swift Crypto (used by SwiftPM) ```cmd -cmake -B S:\b\7\SwiftCrypto ^ +cmake -B S:\b\6\SwiftCrypto ^ -D BUILD_SHARED_LIBS=YES ^ -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ @@ -523,13 +517,13 @@ cmake -B S:\b\7\SwiftCrypto ^ -G Ninja ^ -S S:\swift-crypto -cmake --build S:\b\7\SwiftCrypto +cmake --build S:\b\6\SwiftCrypto ``` #### Swift Collections (used by SwiftPM) ```cmd -cmake -B S:\b\7\SwiftCollections ^ +cmake -B S:\b\6\SwiftCollections ^ -D BUILD_SHARED_LIBS=YES ^ -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ @@ -547,13 +541,13 @@ cmake -B S:\b\7\SwiftCollections ^ -G Ninja ^ -S S:\swift-collections -cmake --build S:\b\7\SwiftCollections +cmake --build S:\b\6\SwiftCollections ``` #### IndexStoreDB (used by SourceKit-LSP) ```cmd -cmake -B S:\b\7\IndexStoreDB ^ +cmake -B S:\b\6\IndexStoreDB ^ -D BUILD_SHARED_LIBS=YES ^ -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ @@ -575,13 +569,13 @@ cmake -B S:\b\7\IndexStoreDB ^ -G Ninja ^ -S S:\indexstore-db -cmake --build S:\b\7\IndexStoreDB +cmake --build S:\b\6\IndexStoreDB ``` -### 8. TSC (without tests) +### 7. TSC (without tests) ```cmd -cmake -B S:\b\8 ^ +cmake -B S:\b\7 ^ -D BUILD_SHARED_LIBS=YES ^ -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ @@ -595,20 +589,20 @@ cmake -B S:\b\8 ^ -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ -D dispatch_DIR=S:\b\3\cmake\modules ^ -D Foundation_DIR=S:\b\4\cmake\modules ^ - -D SwiftSystem_DIR=S:\b\7\SwiftSystem\cmake\modules ^ + -D SwiftSystem_DIR=S:\b\6\SwiftSystem\cmake\modules ^ -D CMAKE_TOOLCHAIN_FILE=S:\vcpkg\scripts\buildsystems\vcpkg.cmake ^ -G Ninja ^ -S S:\swift-tools-support-core -cmake --build S:\b\8 +cmake --build S:\b\7 ``` -### 9. Swift Driver (without tests) +### 8. Swift Driver (without tests) ```cmd -cmake -B S:\b\9 ^ +cmake -B S:\b\8 ^ -D BUILD_SHARED_LIBS=YES ^ -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ @@ -622,22 +616,22 @@ cmake -B S:\b\9 ^ -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ -D dispatch_DIR=S:\b\3\cmake\modules ^ -D Foundation_DIR=S:\b\4\cmake\modules ^ - -D LLBuild_DIR=S:\b\6\cmake\modules ^ - -D Yams_DIR=S:\b\7\Yams\cmake\modules ^ - -D SwiftSystem_DIR=S:\b\7\SwiftSystem\cmake\modules ^ - -D ArgumentParser_DIR=S:\b\7\ArgumentParser\cmake\modules ^ - -D TSC_DIR=S:\b\8\cmake\modules ^ + -D LLBuild_DIR=S:\b\6\LLBuild\cmake\modules ^ + -D Yams_DIR=S:\b\6\Yams\cmake\modules ^ + -D SwiftSystem_DIR=S:\b\6\SwiftSystem\cmake\modules ^ + -D ArgumentParser_DIR=S:\b\6\ArgumentParser\cmake\modules ^ + -D TSC_DIR=S:\b\7\cmake\modules ^ -G Ninja ^ -S S:\swift-driver -cmake --build S:\b\9 +cmake --build S:\b\8 ``` -### 10. SwiftPM (without tests) +### 9. SwiftPM (without tests) ```cmd -cmake -B S:\b\10 ^ +cmake -B S:\b\9 ^ -D BUILD_SHARED_LIBS=YES ^ -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ @@ -651,24 +645,24 @@ cmake -B S:\b\10 ^ -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ -D dispatch_DIR=S:\b\3\cmake\modules ^ -D Foundation_DIR=S:\b\4\cmake\modules ^ - -D LLBuild_DIR=S:\b\6\cmake\modules ^ - -D SwiftSystem_DIR=S:\b\7\SwiftSystem\cmake\modules ^ - -D ArgumentParser_DIR=S:\b\7\ArgumentParser\cmake\modules ^ - -D SwiftCrypto_DIR=S:\b\7\SwiftCrypto\cmake\modules ^ - -D SwiftCollections_DIR=S:\b\7\SwiftCollections\cmake\modules ^ - -D TSC_DIR=S:\b\8\cmake\modules ^ - -D SwiftDriver_DIR=S:\b\9\cmake\modules ^ + -D LLBuild_DIR=S:\b\6\LLBuild\cmake\modules ^ + -D SwiftSystem_DIR=S:\b\6\SwiftSystem\cmake\modules ^ + -D ArgumentParser_DIR=S:\b\6\ArgumentParser\cmake\modules ^ + -D SwiftCrypto_DIR=S:\b\6\SwiftCrypto\cmake\modules ^ + -D SwiftCollections_DIR=S:\b\6\SwiftCollections\cmake\modules ^ + -D TSC_DIR=S:\b\7\cmake\modules ^ + -D SwiftDriver_DIR=S:\b\8\cmake\modules ^ -G Ninja ^ -S S:\swiftpm -cmake --build S:\b\10 +cmake --build S:\b\9 ``` -### 11. SourceKit-LSP (without tests) +### 10. SourceKit-LSP (without tests) ```cmd -cmake -B S:\b\11 ^ +cmake -B S:\b\10 ^ -D BUILD_SHARED_LIBS=YES ^ -D CMAKE_BUILD_TYPE=Release ^ -D CMAKE_INSTALL_PREFIX=S:\b\toolchain\usr ^ @@ -682,18 +676,18 @@ cmake -B S:\b\11 ^ -D CMAKE_Swift_COMPILER=S:/b/1/bin/swiftc.exe ^ -D dispatch_DIR=S:\b\3\cmake\modules ^ -D Foundation_DIR=S:\b\4\cmake\modules ^ - -D LLBuild_DIR=S:\b\6\cmake\modules ^ - -D SwiftSystem_DIR=S:\b\7\SwiftSystem\cmake\modules ^ - -D ArgumentParser_DIR=S:\b\7\ArgumentParser\cmake\modules ^ - -D SwiftCollections_DIR=S:\b\7\SwiftCollections\cmake\modules ^ - -D IndexStoreDB_DIR=S:\b\7\IndexStoreDB\cmake\modules ^ - -D TSC_DIR=S:\b\8\cmake\modules ^ - -D SwiftPM_DIR=S:\b\10\cmake\modules ^ + -D LLBuild_DIR=S:\b\6\LLBuild\cmake\modules ^ + -D SwiftSystem_DIR=S:\b\6\SwiftSystem\cmake\modules ^ + -D ArgumentParser_DIR=S:\b\6\ArgumentParser\cmake\modules ^ + -D SwiftCollections_DIR=S:\b\6\SwiftCollections\cmake\modules ^ + -D IndexStoreDB_DIR=S:\b\6\IndexStoreDB\cmake\modules ^ + -D TSC_DIR=S:\b\7\cmake\modules ^ + -D SwiftPM_DIR=S:\b\9\cmake\modules ^ -G Ninja ^ -S S:\sourcekit-lsp -cmake --build S:\b\11 +cmake --build S:\b\10 ``` ## Use the toolchain and SDK @@ -768,18 +762,18 @@ move /Y S:\b\sdk\usr\lib\swift\windows\*.lib S:\b\sdk\usr\lib\swift\windows\x86_ ### Swift developer tools ```cmd -cmake --build S:\b\6 --target install -cmake --build S:\b\7\Yams --target install -cmake --build S:\b\7\ArgumentParser --target install -cmake --build S:\b\7\SwiftSystem --target install -cmake --build S:\b\7\SwiftCrypto --target install -cmake --build S:\b\7\SwiftCollections --target install -cmake --build S:\b\7\IndexStoreDB --target install +cmake --build S:\b\6\LLBuild --target install +cmake --build S:\b\6\Yams --target install +cmake --build S:\b\6\ArgumentParser --target install +cmake --build S:\b\6\SwiftSystem --target install +cmake --build S:\b\6\SwiftCrypto --target install +cmake --build S:\b\6\SwiftCollections --target install +cmake --build S:\b\6\IndexStoreDB --target install +cmake --build S:\b\7 --target install cmake --build S:\b\8 --target install cmake --build S:\b\9 --target install cmake --build S:\b\10 --target install -cmake --build S:\b\11 --target install -copy /Y S:\b\8\bin\sqlite3.dll S:\b\toolchain\usr\bin +copy /Y S:\b\7\bin\sqlite3.dll S:\b\toolchain\usr\bin ``` To use Swift Driver in place of the old driver (default since Swift 5.7): From 35764a3047ddfd0f0a95342e48b2bd3f92ee4896 Mon Sep 17 00:00:00 2001 From: stevapple Date: Mon, 4 Apr 2022 00:31:09 +0800 Subject: [PATCH 24/25] [WindowsBuild] Update graph to reflect changes --- docs/WindowsBuild.md | 100 ++++++++++++++++++++++--------------------- 1 file changed, 51 insertions(+), 49 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index 1851cf212a84a..27b69d22f6af2 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -131,55 +131,57 @@ If you want to use an existing Swift toolchain to build the core libraries and o Here is a graphical overview of all build steps from this guide. ``` - ┌──────────────────────────────────────┐ ┌────────────────┐ - │ │ │ │ - │ 1. LLVM + Swift compiler │ │ │ - │ │ │ Existing Swift │ - │ ┌────────────────────────────┐ │ or │ toolchain │ - │ │ 2. Swift runtime libraries │ │ │ │ - │ └────────────────────────────┘ │ │ │ - └──────────────────────────────────────┘ └────────────────┘ - │ │ - └─────────┬────────────────────────┘ - │ - ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ - ┌──────────────────────┐ │ - │ │ 3. libdispatch │ - └──────────────────────┘ │ - │ Core ┌──────────────────────┐ - ┌─ │ 4. Foundation │ │─┐ - │ │ libraries └──────────────────────┘ │ - │ ┌──────────────────────┐ │ │ - │ │ │ 5. XCTest │ │ - │ └──────────────────────┘ │ │ - │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ - │ │ - │ ┌──────────────────────────────────────┐ - │ │ │ - │ │ 7. Toolchain dependencies │ -┌────────────────────────────┐ │ │ -│ 6. LLBuild │ ├────┬───────────┬────────┬────────────┤ -└────────────────────────────┘ │Yams│SwiftSystem│ ··· │IndexStoreDB│ - │ └────┴───────────┴────────┴────────────┘ - │ │ - │ │ - │ ┌────────────────────────────┐ - │ │ 8. TSC │ - │ └────────────────────────────┘ - │ │ - └─────────────────────┬────────────────────┘ - │ - ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ - ┌──────────────────────┐ │ - │ │ 9. Swift Driver │ - └──────────────────────┘ │ - │ Developer ┌──────────────────────┐ - │ 10. SwiftPM │ │ - │ tools └──────────────────────┘ - ┌──────────────────────┐ │ - │ │ 11. SourceKit-LSP │ - └──────────────────────┘ │ - └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ +┌──────────────────────────────────────┐ ┌────────────────┐ +│ │ │ │ +│ 1. LLVM + Swift compiler │ │ │ +│ │ │ Existing Swift │ +│ ┌────────────────────────────┐ │ or │ toolchain │ +│ │ 2. Swift runtime libraries │ │ │ │ +│ └────────────────────────────┘ │ │ │ +└──────────────────────────────────────┘ └────────────────┘ + │ │ + └───────────┬──────────────────────┘ + │ + ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ + ┌──────────────────────┐ │ + │ │ 3. libdispatch │ + └──────────────────────┘ │ + │ Core ┌──────────────────────┐ + │ 4. Foundation │ │ + │ libraries └──────────────────────┘ + ┌──────────────────────┐ │ + │ │ 5. XCTest │ + └──────────────────────┘ │ + └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ + │ + │ + │ + ┌───────────────────────────────────────────────────┐ + │ │ + │ 6. Developer tool dependencies │ + │ │ + ├─────────┬──────┬─────────────┬─────┬──────────────┤ + │ LLBuild │ Yams │ SwiftSystem │ ··· │ IndexStoreDB │ + └─────────┴──────┴─────────────┴─────┴──────────────┘ + │ + │ + ┌────────────────────────────┐ + │ 7. TSC │ + └────────────────────────────┘ + │ + │ + │ + ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ + ┌──────────────────────┐ │ + │ │ 8. Swift Driver │ + └──────────────────────┘ │ + │ Developer ┌──────────────────────┐ + │ 9. SwiftPM │ │ + │ tools └──────────────────────┘ + ┌──────────────────────┐ │ + │ │ 10. SourceKit-LSP │ + └──────────────────────┘ │ + └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ``` ### 1. Swift compiler From 3cca99db7928619fcce8df1342ab0298b16697ba Mon Sep 17 00:00:00 2001 From: stevapple Date: Mon, 4 Apr 2022 00:38:30 +0800 Subject: [PATCH 25/25] [WindowsBuild] Drop minimal build instruction --- docs/WindowsBuild.md | 28 ---------------------------- 1 file changed, 28 deletions(-) diff --git a/docs/WindowsBuild.md b/docs/WindowsBuild.md index 27b69d22f6af2..4b32d7ef77b7b 100644 --- a/docs/WindowsBuild.md +++ b/docs/WindowsBuild.md @@ -88,34 +88,6 @@ mklink "%VCToolsInstallDir%\include\visualc.apinotes" S:\swift\stdlib\public\Pla > **WARNING:** Creating the above links usually requires administrator privileges. The quick and easy way to do this is to open a second developer prompt by right clicking whatever shortcut you used to open the first one, choosing "More > Run As Administrator", and pasting the above commands into the resulting window. You can then close the privileged prompt; this is the only step which requires elevation. -## Build a minimal Swift toolchain - -A minimal Swift toolchain does not include optional features or SwiftPM stuffs, and may be useful for playing with the compiler or language itself. - -```cmd -cmake -B S:\b\1 ^ - -C S:\swift\cmake\caches\Windows-x86_64.cmake ^ - -D CMAKE_BUILD_TYPE=Release ^ - -D CMAKE_INSTALL_PREFIX=C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr ^ - -D CMAKE_C_COMPILER=cl.exe ^ - -D CMAKE_C_FLAGS="/GS- /Oy /Gw /Gy /source-charset:utf-8 /execution-charset:utf-8" ^ - -D CMAKE_CXX_COMPILER=cl.exe ^ - -D CMAKE_CXX_FLAGS="/GS- /Oy /Gw /Gy /utf-8" ^ - -D CMAKE_MT=mt ^ - -D CMAKE_EXE_LINKER_FLAGS="/INCREMENTAL:NO" ^ - -D CMAKE_SHARED_LINKER_FLAGS="/INCREMENTAL:NO" ^ - -D LLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-windows-msvc ^ - -D LLVM_APPEND_VC_REV=NO ^ - -D LLVM_EXTERNAL_CMARK_SOURCE_DIR=S:\cmark ^ - -D LLVM_EXTERNAL_SWIFT_SOURCE_DIR=S:\swift ^ - -D SWIFT_PATH_TO_LIBDISPATCH_SOURCE=S:\swift-corelibs-libdispatch ^ - -G Ninja ^ - -S S:\llvm-project\llvm - -cmake --build S:\b\1 -cmake --build S:\b\1 --target install -``` - ## Build for development The following guide will get you through the building process of a complete Swift debug toolchain.