From 19a333ad0fe9a892ea221aff6bb660524c777233 Mon Sep 17 00:00:00 2001 From: onur-ozkan Date: Fri, 14 Mar 2025 23:08:21 +0300 Subject: [PATCH 1/4] split `mingw-check` into two Signed-off-by: onur-ozkan --- src/tests/ci.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/tests/ci.md b/src/tests/ci.md index 825be11c8..d1bd6ac91 100644 --- a/src/tests/ci.md +++ b/src/tests/ci.md @@ -66,8 +66,8 @@ kinds of builds (sets of jobs). ### Pull Request builds After each push to a pull request, a set of `pr` jobs are executed. Currently, -these execute the `x86_64-gnu-llvm-X`, `x86_64-gnu-tools`, `mingw-check` and -`mingw-check-tidy` jobs, all running on Linux. These execute a relatively short +these execute the `x86_64-gnu-llvm-X`, `x86_64-gnu-tools`, `mingw-check-1`, `mingw-check-2` +and `mingw-check-tidy` jobs, all running on Linux. These execute a relatively short (~30 minutes) and lightweight test suite that should catch common issues. More specifically, they run a set of lints, they try to perform a cross-compile check build to Windows mingw (without producing any artifacts) and they test the From 09eb3b3f783db3a7d440582264c95ef8da66d533 Mon Sep 17 00:00:00 2001 From: onur-ozkan Date: Fri, 14 Mar 2025 08:59:23 +0300 Subject: [PATCH 2/4] update dev guidelines Signed-off-by: onur-ozkan --- .../bootstrapping/what-bootstrapping-does.md | 86 ++++++++----------- src/building/how-to-build-and-run.md | 14 ++- src/tests/ci.md | 2 +- 3 files changed, 41 insertions(+), 61 deletions(-) diff --git a/src/building/bootstrapping/what-bootstrapping-does.md b/src/building/bootstrapping/what-bootstrapping-does.md index a2930b3e4..e5d1cd8f1 100644 --- a/src/building/bootstrapping/what-bootstrapping-does.md +++ b/src/building/bootstrapping/what-bootstrapping-does.md @@ -45,13 +45,13 @@ compiler. ```mermaid graph TD - s0c["stage0 compiler (1.63)"]:::downloaded -->|A| s0l("stage0 std (1.64)"):::with-s0c; + s0c["stage0 compiler (1.86.0-beta.1)"]:::downloaded -->|A| s0l("stage0 std (1.86.0-beta.1)"):::downloaded; s0c & s0l --- stepb[ ]:::empty; - stepb -->|B| s0ca["stage0 compiler artifacts (1.64)"]:::with-s0c; - s0ca -->|copy| s1c["stage1 compiler (1.64)"]:::with-s0c; - s1c -->|C| s1l("stage1 std (1.64)"):::with-s1c; + stepb -->|B| s0ca["stage0 compiler artifacts (1.87.0-dev)"]:::with-s0c; + s0ca -->|copy| s1c["stage1 compiler (1.87.0-dev)"]:::with-s0c; + s1c -->|C| s1l("stage1 std (1.87.0-dev)"):::with-s1c; s1c & s1l --- stepd[ ]:::empty; - stepd -->|D| s1ca["stage1 compiler artifacts (1.64)"]:::with-s1c; + stepd -->|D| s1ca["stage1 compiler artifacts (1.87.0-dev)"]:::with-s1c; s1ca -->|copy| s2c["stage2 compiler"]:::with-s1c; classDef empty width:0px,height:0px; @@ -62,19 +62,22 @@ graph TD ### Stage 0: the pre-compiled compiler -The stage0 compiler is usually the current _beta_ `rustc` compiler and its +The stage0 compiler is by default the very recent _beta_ `rustc` compiler and its associated dynamic libraries, which `./x.py` will download for you. (You can -also configure `./x.py` to use something else.) +also configure `./x.py` to change stage0 to something else.) -The stage0 compiler is then used only to compile [`src/bootstrap`], -[`library/std`], and [`compiler/rustc`]. When assembling the libraries and -binaries that will become the stage1 `rustc` compiler, the freshly compiled -`std` and `rustc` are used. There are two concepts at play here: a compiler -(with its set of dependencies) and its 'target' or 'object' libraries (`std` and -`rustc`). Both are staged, but in a staggered manner. +The stage0 compiler is then used only to compile [`src/bootstrap`] and [`compiler/rustc`]. +When assembling the libraries and binaries that will become the stage1 `rustc` compiler, +the freshly compiled `rustc` and beta `std` are used. + +Note that to build the stage1 compiler we use the precompiled beta compiler and beta std from stage0. +Therefore, to use a compiler with a std that is freshly built from the tree, you need to build the +stage2 compiler. + +There are two concepts at play here: a compiler (with its set of dependencies) and its +'target' or 'object' libraries (`std` and `rustc`). Both are staged, but in a staggered manner. [`compiler/rustc`]: https://github.com/rust-lang/rust/tree/master/compiler/rustc -[`library/std`]: https://github.com/rust-lang/rust/tree/master/library/std [`src/bootstrap`]: https://github.com/rust-lang/rust/tree/master/src/bootstrap ### Stage 1: from current code, by an earlier compiler @@ -84,16 +87,14 @@ The rustc source code is then compiled with the `stage0` compiler to produce the ### Stage 2: the truly current compiler -We then rebuild our `stage1` compiler with itself to produce the `stage2` +We then rebuild our `stage1` compiler with in-tree std to produce the `stage2` compiler. -In theory, the `stage1` compiler is functionally identical to the `stage2` -compiler, but in practice there are subtle differences. In particular, the -`stage1` compiler itself was built by `stage0` and hence not by the source in -your working directory. This means that the ABI generated by the `stage0` -compiler may not match the ABI that would have been made by the `stage1` -compiler, which can cause problems for dynamic libraries, tests, and tools using -`rustc_private`. +The `stage1` compiler itself was built by precompiled `stage0` compiler and std +and hence not by the source in your working directory. This means that the ABI +generated by the `stage0` compiler may not match the ABI that would have been made +by the `stage1` compiler, which can cause problems for dynamic libraries, tests +and tools using `rustc_private`. Note that the `proc_macro` crate avoids this issue with a `C` FFI layer called `proc_macro::bridge`, allowing it to be used with `stage1`. @@ -101,9 +102,10 @@ Note that the `proc_macro` crate avoids this issue with a `C` FFI layer called The `stage2` compiler is the one distributed with `rustup` and all other install methods. However, it takes a very long time to build because one must first build the new compiler with an older compiler and then use that to build the new -compiler with itself. For development, you usually only want the `stage1` -compiler, which you can build with `./x build library`. See [Building the -compiler](../how-to-build-and-run.html#building-the-compiler). +compiler with itself. + +For development, you usually only want to use `--stage 1` flag to build things. +See [Building the compiler](../how-to-build-and-run.html#building-the-compiler). ### Stage 3: the same-result test @@ -114,10 +116,11 @@ something has broken. ### Building the stages The script [`./x`] tries to be helpful and pick the stage you most likely meant -for each subcommand. These defaults are as follows: +for each subcommand. Here are some `x` commands with their default stages: -- `check`: `--stage 0` -- `doc`: `--stage 0` +- `check`: `--stage 1` +- `clippy`: `--stage 1` +- `doc`: `--stage 1` - `build`: `--stage 1` - `test`: `--stage 1` - `dist`: `--stage 2` @@ -208,9 +211,6 @@ include, but are not limited to: ### Building vs. running -Note that `build --stage N compiler/rustc` **does not** build the stage N -compiler: instead it builds the stage N+1 compiler _using_ the stage N compiler. - In short, _stage 0 uses the `stage0` compiler to create `stage0` artifacts which will later be uplifted to be the stage1 compiler_. @@ -268,23 +268,6 @@ However, when cross-compiling, `stage1` `std` will only run on the host. So the (See in the table how `stage2` only builds non-host `std` targets). -### Why does only libstd use `cfg(bootstrap)`? - -For docs on `cfg(bootstrap)` itself, see [Complications of -Bootstrapping](#complications-of-bootstrapping). - -The `rustc` generated by the `stage0` compiler is linked to the freshly-built -`std`, which means that for the most part only `std` needs to be `cfg`-gated, so -that `rustc` can use features added to `std` immediately after their addition, -without need for them to get into the downloaded `beta` compiler. - -Note this is different from any other Rust program: `stage1` `rustc` is built by -the _beta_ compiler, but using the _master_ version of `libstd`! - -The only time `rustc` uses `cfg(bootstrap)` is when it adds internal lints that -use diagnostic items, or when it uses unstable library features that were -recently changed. - ### What is a 'sysroot'? When you build a project with `cargo`, the build artifacts for dependencies are @@ -459,7 +442,6 @@ compiler itself uses to run. These aren't actually used by artifacts the new compiler generates. This step also copies the `rustc` and `rustdoc` binaries we generated into `build/$HOST/stage/bin`. -The `stage1/bin/rustc` is a fully functional compiler, but it doesn't yet have -any libraries to link built binaries or libraries to. The next 3 steps will -provide those libraries for it; they are mostly equivalent to constructing the -`stage1/bin` compiler so we don't go through them individually here. +The `stage1/bin/rustc` is a fully functional compiler built with the beta compiler and std. +To use a compiler built entirely from source with the in-tree compiler and std, you need to build +the stage2 compiler, which is compiled using the stage1 compiler and std. diff --git a/src/building/how-to-build-and-run.md b/src/building/how-to-build-and-run.md index c3c1c41e3..a6e9d0b73 100644 --- a/src/building/how-to-build-and-run.md +++ b/src/building/how-to-build-and-run.md @@ -217,7 +217,6 @@ probably the best "go to" command for building a local compiler: This may *look* like it only builds the standard library, but that is not the case. What this command does is the following: -- Build `std` using the stage0 compiler - Build `rustc` using the stage0 compiler - This produces the stage1 compiler - Build `std` using the stage1 compiler @@ -241,8 +240,7 @@ build. The **full** `rustc` build (what you get with `./x build --stage 2 compiler/rustc`) has quite a few more steps: - Build `rustc` with the stage1 compiler. - - The resulting compiler here is called the "stage2" compiler. -- Build `std` with stage2 compiler. + - The resulting compiler here is called the "stage2" compiler, which uses stage1 std from the previous command. - Build `librustdoc` and a bunch of other things with the stage2 compiler. You almost never need to do this. @@ -250,14 +248,14 @@ You almost never need to do this. ### Build specific components If you are working on the standard library, you probably don't need to build -the compiler unless you are planning to use a recently added nightly feature. -Instead, you can just build using the bootstrap compiler. +every other default component. Instead, you can build a specific component by +providing its name, like this: ```bash -./x build --stage 0 library +./x build --stage 1 library ``` -If you choose the `library` profile when running `x setup`, you can omit `--stage 0` (it's the +If you choose the `library` profile when running `x setup`, you can omit `--stage 1` (it's the default). ## Creating a rustup toolchain @@ -271,7 +269,7 @@ you will likely need to build at some point; for example, if you want to run the entire test suite). ```bash -rustup toolchain link stage0 build/host/stage0-sysroot # beta compiler + stage0 std +rustup toolchain link stage0 build/host/stage0-sysroot # beta compiler + beta std rustup toolchain link stage1 build/host/stage1 rustup toolchain link stage2 build/host/stage2 ``` diff --git a/src/tests/ci.md b/src/tests/ci.md index d1bd6ac91..8f259f270 100644 --- a/src/tests/ci.md +++ b/src/tests/ci.md @@ -68,7 +68,7 @@ kinds of builds (sets of jobs). After each push to a pull request, a set of `pr` jobs are executed. Currently, these execute the `x86_64-gnu-llvm-X`, `x86_64-gnu-tools`, `mingw-check-1`, `mingw-check-2` and `mingw-check-tidy` jobs, all running on Linux. These execute a relatively short -(~30 minutes) and lightweight test suite that should catch common issues. More +(~40 minutes) and lightweight test suite that should catch common issues. More specifically, they run a set of lints, they try to perform a cross-compile check build to Windows mingw (without producing any artifacts) and they test the compiler using a *system* version of LLVM. Unfortunately, it would take too many From 257e73f3a30d70ef51509733258b36948a903039 Mon Sep 17 00:00:00 2001 From: onur-ozkan Date: Fri, 14 Mar 2025 23:19:55 +0300 Subject: [PATCH 3/4] improve comments and docs Signed-off-by: onur-ozkan --- .../bootstrapping/what-bootstrapping-does.md | 23 ++++---- src/building/how-to-build-and-run.md | 1 - src/building/new-target.md | 2 +- src/building/suggested.md | 54 ++++--------------- 4 files changed, 21 insertions(+), 59 deletions(-) diff --git a/src/building/bootstrapping/what-bootstrapping-does.md b/src/building/bootstrapping/what-bootstrapping-does.md index e5d1cd8f1..2793ad438 100644 --- a/src/building/bootstrapping/what-bootstrapping-does.md +++ b/src/building/bootstrapping/what-bootstrapping-does.md @@ -66,13 +66,12 @@ The stage0 compiler is by default the very recent _beta_ `rustc` compiler and it associated dynamic libraries, which `./x.py` will download for you. (You can also configure `./x.py` to change stage0 to something else.) -The stage0 compiler is then used only to compile [`src/bootstrap`] and [`compiler/rustc`]. -When assembling the libraries and binaries that will become the stage1 `rustc` compiler, -the freshly compiled `rustc` and beta `std` are used. +The precompiled stage0 compiler is then used only to compile [`src/bootstrap`] and [`compiler/rustc`] +with precompiled stage0 std. -Note that to build the stage1 compiler we use the precompiled beta compiler and beta std from stage0. -Therefore, to use a compiler with a std that is freshly built from the tree, you need to build the -stage2 compiler. +Note that to build the stage1 compiler we use the precompiled stage0 compiler and std. +Therefore, to use a compiler with a std that is freshly built from the tree, you need to +build the stage2 compiler. There are two concepts at play here: a compiler (with its set of dependencies) and its 'target' or 'object' libraries (`std` and `rustc`). Both are staged, but in a staggered manner. @@ -87,7 +86,7 @@ The rustc source code is then compiled with the `stage0` compiler to produce the ### Stage 2: the truly current compiler -We then rebuild our `stage1` compiler with in-tree std to produce the `stage2` +We then rebuild the compiler using `stage1` compiler with in-tree std to produce the `stage2` compiler. The `stage1` compiler itself was built by precompiled `stage0` compiler and std @@ -194,8 +193,8 @@ include, but are not limited to: without building `rustc` from source ('build with `stage0`, then test the artifacts'). If you're working on the standard library, this is normally the test command you want. -- `./x build --stage 0` means to build with the beta `rustc`. -- `./x doc --stage 0` means to document using the beta `rustdoc`. +- `./x build --stage 0` means to build with the stage0 `rustc`. +- `./x doc --stage 0` means to document using the stage0 `rustdoc`. #### Examples of what *not* to do @@ -442,6 +441,6 @@ compiler itself uses to run. These aren't actually used by artifacts the new compiler generates. This step also copies the `rustc` and `rustdoc` binaries we generated into `build/$HOST/stage/bin`. -The `stage1/bin/rustc` is a fully functional compiler built with the beta compiler and std. -To use a compiler built entirely from source with the in-tree compiler and std, you need to build -the stage2 compiler, which is compiled using the stage1 compiler and std. +The `stage1/bin/rustc` is a fully functional compiler built with stage0 (precompiled) compiler and std. +To use a compiler built entirely from source with the in-tree compiler and std, you need to build the +stage2 compiler, which is compiled using the stage1 (in-tree) compiler and std. diff --git a/src/building/how-to-build-and-run.md b/src/building/how-to-build-and-run.md index a6e9d0b73..c4783002b 100644 --- a/src/building/how-to-build-and-run.md +++ b/src/building/how-to-build-and-run.md @@ -269,7 +269,6 @@ you will likely need to build at some point; for example, if you want to run the entire test suite). ```bash -rustup toolchain link stage0 build/host/stage0-sysroot # beta compiler + beta std rustup toolchain link stage1 build/host/stage1 rustup toolchain link stage2 build/host/stage2 ``` diff --git a/src/building/new-target.md b/src/building/new-target.md index 09ffbe8c8..8d323ba96 100644 --- a/src/building/new-target.md +++ b/src/building/new-target.md @@ -85,7 +85,7 @@ Look for existing targets to use as examples. After adding your target to the `rustc_target` crate you may want to add `core`, `std`, ... with support for your new target. In that case you will probably need access to some `target_*` cfg. Unfortunately when building with -stage0 (the beta compiler), you'll get an error that the target cfg is +stage0 (a precompiled compiler), you'll get an error that the target cfg is unexpected because stage0 doesn't know about the new target specification and we pass `--check-cfg` in order to tell it to check. diff --git a/src/building/suggested.md b/src/building/suggested.md index f8a28b7f2..333554c8a 100644 --- a/src/building/suggested.md +++ b/src/building/suggested.md @@ -310,51 +310,15 @@ lets you use `cargo fmt`. [the section on vscode]: suggested.md#configuring-rust-analyzer-for-rustc [the section on rustup]: how-to-build-and-run.md?highlight=rustup#creating-a-rustup-toolchain -## Faster builds with `--keep-stage`. - -Sometimes just checking whether the compiler builds is not enough. A common -example is that you need to add a `debug!` statement to inspect the value of -some state or better understand the problem. In that case, you don't really need -a full build. By bypassing bootstrap's cache invalidation, you can often get -these builds to complete very fast (e.g., around 30 seconds). The only catch is -this requires a bit of fudging and may produce compilers that don't work (but -that is easily detected and fixed). - -The sequence of commands you want is as follows: - -- Initial build: `./x build library` - - As [documented previously], this will build a functional stage1 compiler as - part of running all stage0 commands (which include building a `std` - compatible with the stage1 compiler) as well as the first few steps of the - "stage 1 actions" up to "stage1 (sysroot stage1) builds std". -- Subsequent builds: `./x build library --keep-stage 1` - - Note that we added the `--keep-stage 1` flag here - -[documented previously]: ./how-to-build-and-run.md#building-the-compiler - -As mentioned, the effect of `--keep-stage 1` is that we just _assume_ that the -old standard library can be re-used. If you are editing the compiler, this is -almost always true: you haven't changed the standard library, after all. But -sometimes, it's not true: for example, if you are editing the "metadata" part of -the compiler, which controls how the compiler encodes types and other states -into the `rlib` files, or if you are editing things that wind up in the metadata -(such as the definition of the MIR). - -**The TL;DR is that you might get weird behavior from a compile when using -`--keep-stage 1`** -- for example, strange [ICEs](../appendix/glossary.html#ice) -or other panics. In that case, you should simply remove the `--keep-stage 1` -from the command and rebuild. That ought to fix the problem. - -You can also use `--keep-stage 1` when running tests. Something like this: - -- Initial test run: `./x test tests/ui` -- Subsequent test run: `./x test tests/ui --keep-stage 1` - -### Iterating the standard library with `--keep-stage` - -If you are making changes to the standard library, you can use `./x build ---keep-stage 0 library` to iteratively rebuild the standard library without -rebuilding the compiler. +## Faster Builds with CI-rustc + +If you are not working on the compiler, you often don't need to build the compiler tree. +For example, you can skip building the compiler and only build the `library` tree or the +tools under `src/tools`. To achieve that, you have to enable this by setting the `download-rustc` +option in your configuration. This tells bootstrap to use the latest nightly compiler for `stage > 0` +steps, meaning it will have two precompiled compilers: stage0 compiler and `download-rustc` compiler +for `stage > 0` steps. This way, it will never need to build the in-tree compiler. As a result, your +build time will be significantly reduced by not building the in-tree compiler. ## Using incremental compilation From 91be3cb103b4e316bcac7bc7d207d3631a21ff11 Mon Sep 17 00:00:00 2001 From: Jieyou Xu Date: Tue, 3 Jun 2025 20:16:00 +0800 Subject: [PATCH 4/4] Preparing for merge from rustc --- rust-version | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rust-version b/rust-version index b1e9eec52..8b48bd518 100644 --- a/rust-version +++ b/rust-version @@ -1 +1 @@ -99e7c15e81385b38a8186b51edc4577d5d7b5bdd +c68032fd4c442d275f4daa571ba19c076106b490