From 912575e174f41c19e56a9b34d0ec8a81e8775373 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Ber=C3=A1nek?= Date: Tue, 11 Feb 2025 13:21:40 +0100 Subject: [PATCH 1/5] Update documentation --- src/building/optimized-build.md | 2 +- src/tests/ci.md | 6 +++--- src/tests/docker.md | 9 +++++++++ 3 files changed, 13 insertions(+), 4 deletions(-) diff --git a/src/building/optimized-build.md b/src/building/optimized-build.md index 8feda5982..f8ca1d0dc 100644 --- a/src/building/optimized-build.md +++ b/src/building/optimized-build.md @@ -126,4 +126,4 @@ Here is an example of how can `opt-dist` be used locally (outside of CI): [`Environment`]: https://github.com/rust-lang/rust/blob/ee451f8faccf3050c76cdcd82543c917b40c7962/src/tools/opt-dist/src/environment.rs#L5 > Note: if you want to run the actual CI pipeline, instead of running `opt-dist` locally, -> you can execute `python3 src/ci/github-actions/ci.py run-local dist-x86_64-linux`. +> you can execute `cargo run --manifest-path src/ci/citool/Cargo.toml run-local dist-x86_64-linux`. diff --git a/src/tests/ci.md b/src/tests/ci.md index a4b22392f..ae6adb678 100644 --- a/src/tests/ci.md +++ b/src/tests/ci.md @@ -28,7 +28,7 @@ Our CI is primarily executed on [GitHub Actions], with a single workflow defined in [`.github/workflows/ci.yml`], which contains a bunch of steps that are unified for all CI jobs that we execute. When a commit is pushed to a corresponding branch or a PR, the workflow executes the -[`src/ci/github-actions/ci.py`] script, which dynamically generates the specific CI +[`src/ci/citool`] crate, which dynamically generates the specific CI jobs that should be executed. This script uses the [`jobs.yml`] file as an input, which contains a declarative configuration of all our CI jobs. @@ -299,7 +299,7 @@ platform’s custom [Docker container]. This has a lot of advantages for us: - We can avoid reinstalling tools (like QEMU or the Android emulator) every time thanks to Docker image caching. - Users can run the same tests in the same environment locally by just running - `python3 src/ci/github-actions/ci.py run-local `, which is awesome to debug failures. Note that there are only linux docker images available locally due to licensing and + `cargo run --manifest-path src/ci/citool/Cargo.toml run-local `, which is awesome to debug failures. Note that there are only linux docker images available locally due to licensing and other restrictions. The docker images prefixed with `dist-` are used for building artifacts while @@ -443,7 +443,7 @@ this: [GitHub Actions]: https://github.com/rust-lang/rust/actions [`jobs.yml`]: https://github.com/rust-lang/rust/blob/master/src/ci/github-actions/jobs.yml [`.github/workflows/ci.yml`]: https://github.com/rust-lang/rust/blob/master/.github/workflows/ci.yml -[`src/ci/github-actions/ci.py`]: https://github.com/rust-lang/rust/blob/master/src/ci/github-actions/ci.py +[`src/ci/citool`]: https://github.com/rust-lang/rust/blob/master/src/ci/citool [rust-lang-ci]: https://github.com/rust-lang-ci/rust/actions [bors]: https://github.com/bors [homu]: https://github.com/rust-lang/homu diff --git a/src/tests/docker.md b/src/tests/docker.md index 2ca08d421..a8a388ef9 100644 --- a/src/tests/docker.md +++ b/src/tests/docker.md @@ -53,6 +53,15 @@ Some additional notes about using the interactive mode: containers. With the container name, run `docker exec -it /bin/bash` where `` is the container name like `4ba195e95cef`. +The approach described above is a relatively low-level interface for running the Docker images +directly. If you want to run a full CI Linux job locally with Docker, in a way that is as close to CI as possible, you can use the following command: + +```bash +cargo run --manifest-path src/ci/citool/Cargo.toml run-local +# For example: +cargo run --manifest-path src/ci/citool/Cargo.toml run-local dist-x86_64-linux-alt +``` + [Docker]: https://www.docker.com/ [`src/ci/docker`]: https://github.com/rust-lang/rust/tree/master/src/ci/docker [`src/ci/docker/run.sh`]: https://github.com/rust-lang/rust/blob/master/src/ci/docker/run.sh From 8044303cbfaca1ca145bb8002d478af970e0835a Mon Sep 17 00:00:00 2001 From: Noratrieb <48135649+Noratrieb@users.noreply.github.com> Date: Sat, 18 Jan 2025 18:19:42 +0100 Subject: [PATCH 2/5] Support raw-dylib link kind on ELF raw-dylib is a link kind that allows rustc to link against a library without having any library files present. This currently only exists on Windows. rustc will take all the symbols from raw-dylib link blocks and put them in an import library, where they can then be resolved by the linker. While import libraries don't exist on ELF, it would still be convenient to have this same functionality. Not having the libraries present at build-time can be convenient for several reasons, especially cross-compilation. With raw-dylib, code linking against a library can be cross-compiled without needing to have these libraries available on the build machine. If the libc crate makes use of this, it would allow cross-compilation without having any libc available on the build machine. This is not yet possible with this implementation, at least against libc's like glibc that use symbol versioning. The raw-dylib kind could be extended with support for symbol versioning in the future. This implementation is very experimental and I have not tested it very well. I have tested it for a toy example and the lz4-sys crate, where it was able to successfully link a binary despite not having a corresponding library at build-time. --- src/tests/directives.md | 1 + 1 file changed, 1 insertion(+) diff --git a/src/tests/directives.md b/src/tests/directives.md index 00bb2bc4d..af9cb85d2 100644 --- a/src/tests/directives.md +++ b/src/tests/directives.md @@ -142,6 +142,7 @@ Some examples of `X` in `ignore-X` or `only-X`: matches that target as well as the emscripten targets. - Pointer width: `32bit`, `64bit` - Endianness: `endian-big` +- Binary format: `elf` - Stage: `stage0`, `stage1`, `stage2` - Channel: `stable`, `beta` - When cross compiling: `cross-compile` From 6a61f6f721c0e1999a04ac0f7a3f37af5a47eb34 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=AE=B8=E6=9D=B0=E5=8F=8B=20Jieyou=20Xu=20=28Joe=29?= <39484203+jieyouxu@users.noreply.github.com> Date: Wed, 5 Feb 2025 21:40:57 +0800 Subject: [PATCH 3/5] rustc-dev-guide: remove mentions of legacy `Makefile` run-make infra And remove outdated requirements to run `run-make` tests on Windows. --- src/tests/compiletest.md | 30 +----------------------------- src/tests/directives.md | 7 +------ src/tests/running.md | 25 ------------------------- 3 files changed, 2 insertions(+), 60 deletions(-) diff --git a/src/tests/compiletest.md b/src/tests/compiletest.md index 459c08290..2905e470f 100644 --- a/src/tests/compiletest.md +++ b/src/tests/compiletest.md @@ -74,8 +74,7 @@ The following test suites are available, with links for more information: ### General purpose test suite -[`run-make`](#run-make-tests) are general purpose tests using Rust programs (or -Makefiles (legacy)). +[`run-make`](#run-make-tests) are general purpose tests using Rust programs. ### Rustdoc test suites @@ -396,14 +395,6 @@ your test, causing separate files to be generated for 32bit and 64bit systems. ### `run-make` tests -> **Note on phasing out `Makefile`s** -> -> We are planning to migrate all existing Makefile-based `run-make` tests -> to Rust programs. You should not be adding new Makefile-based `run-make` -> tests. -> -> See . - The tests in [`tests/run-make`] are general-purpose tests using Rust *recipes*, which are small programs (`rmake.rs`) allowing arbitrary Rust code such as `rustc` invocations, and is supported by a [`run_make_support`] library. Using @@ -424,11 +415,6 @@ Compiletest directives like `//@ only-` or `//@ ignore-` are supported in `rmake.rs`, like in UI tests. However, revisions or building auxiliary via directives are not currently supported. -Two `run-make` tests are ported over to Rust recipes as examples: - -- -- - #### Quickly check if `rmake.rs` tests can be compiled You can quickly check if `rmake.rs` tests can be compiled without having to @@ -481,20 +467,6 @@ Then add a corresponding entry to `"rust-analyzer.linkedProjects"` ], ``` -#### Using Makefiles (legacy) - -
-You should avoid writing new Makefile-based `run-make` tests. -
- -Each test should be in a separate directory with a `Makefile` indicating the -commands to run. - -There is a [`tools.mk`] Makefile which you can include which provides a bunch of -utilities to make it easier to run commands and compare outputs. Take a look at -some of the other tests for some examples on how to get started. - -[`tools.mk`]: https://github.com/rust-lang/rust/blob/master/tests/run-make/tools.mk [`tests/run-make`]: https://github.com/rust-lang/rust/tree/master/tests/run-make [`run_make_support`]: https://github.com/rust-lang/rust/tree/master/src/tools/run-make-support diff --git a/src/tests/directives.md b/src/tests/directives.md index 00bb2bc4d..c72c7d7d8 100644 --- a/src/tests/directives.md +++ b/src/tests/directives.md @@ -6,10 +6,7 @@ FIXME(jieyouxu) completely revise this chapter. --> -Directives are special comments that tell compiletest how to build and interpret -a test. They must appear before the Rust source in the test. They may also -appear in `rmake.rs` or legacy Makefiles for [run-make -tests](compiletest.md#run-make-tests). +Directives are special comments that tell compiletest how to build and interpret a test. They must appear before the Rust source in the test. They may also appear in `rmake.rs` [run-make tests](compiletest.md#run-make-tests). They are normally put after the short comment that explains the point of this test. Compiletest test suites use `//@` to signal that a comment is a directive. @@ -221,8 +218,6 @@ The following directives will check LLVM support: [`aarch64-gnu-debug`]), which only runs a subset of `run-make` tests. Other tests with this directive will not run at all, which is usually not what you want. - - Notably, the [`aarch64-gnu-debug`] CI job *currently* only runs `run-make` - tests which additionally contain `clang` in their test name. See also [Debuginfo tests](compiletest.md#debuginfo-tests) for directives for ignoring debuggers. diff --git a/src/tests/running.md b/src/tests/running.md index 6ce650923..9ddf0afee 100644 --- a/src/tests/running.md +++ b/src/tests/running.md @@ -238,30 +238,6 @@ This is much faster, but doesn't always work. For example, some tests include directives that specify specific compiler flags, or which rely on other crates, and they may not run the same without those options. -## Running `run-make` tests - -### Windows - -Running the `run-make` test suite on Windows is a currently bit more involved. -There are numerous prerequisites and environmental requirements: - -- Install msys2: -- Specify `MSYS2_PATH_TYPE=inherit` in `msys2.ini` in the msys2 installation directory, run the - following with `MSYS2 MSYS`: - - `pacman -Syuu` - - `pacman -S make` - - `pacman -S diffutils` - - `pacman -S binutils` - - `./x test run-make` (`./x test tests/run-make` doesn't work) - -There is [on-going work][port-run-make] to not rely on `Makefile`s in the -run-make test suite. Once this work is completed, you can run the entire -`run-make` test suite on native Windows inside `cmd` or `PowerShell` without -needing to install and use MSYS2. As of Oct 2024, it is -already possible to run the vast majority of the `run-make` test suite outside -of MSYS2, but there will be failures for the tests that still use `Makefile`s -due to not finding `make`. - ## Running tests on a remote machine Tests may be run on a remote machine (e.g. to test builds for a different @@ -406,4 +382,3 @@ If you encounter bugs or problems, don't hesitate to open issues on the repository](https://github.com/rust-lang/rustc_codegen_gcc/). [`tests/ui`]: https://github.com/rust-lang/rust/tree/master/tests/ui -[port-run-make]: https://github.com/rust-lang/rust/issues/121876 From 4801165af51ba8fde444d0821002b6e99f3e4a9a Mon Sep 17 00:00:00 2001 From: Zalathar Date: Mon, 3 Mar 2025 20:17:26 +1100 Subject: [PATCH 4/5] Remove some unnecessary aliases from `rustc_data_structures::sync` With the removal of `cfg(parallel_compiler)`, these are always shared references and `std::sync::OnceLock`. --- src/parallel-rustc.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/parallel-rustc.md b/src/parallel-rustc.md index c5b70706a..690fb19c9 100644 --- a/src/parallel-rustc.md +++ b/src/parallel-rustc.md @@ -46,10 +46,8 @@ are implemented differently depending on whether `parallel-compiler` is true. | data structure | parallel | non-parallel | | -------------------------------- | --------------------------------------------------- | ------------ | -| OnceCell | std::sync::OnceLock | std::cell::OnceCell | | Lock\ | (parking_lot::Mutex\) | (std::cell::RefCell) | | RwLock\ | (parking_lot::RwLock\) | (std::cell::RefCell) | -| MTRef<'a, T> | &'a T | &'a mut T | | MTLock\ | (Lock\) | (T) | | ReadGuard | parking_lot::RwLockReadGuard | std::cell::Ref | | MappedReadGuard | parking_lot::MappedRwLockReadGuard | std::cell::Ref | From 257f03dc44751d2873b28f96cc13b89fc197c184 Mon Sep 17 00:00:00 2001 From: The rustc-dev-guide Cronjob Bot Date: Thu, 6 Mar 2025 04:02:41 +0000 Subject: [PATCH 5/5] Preparing for merge from rustc --- rust-version | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rust-version b/rust-version index ce21bb8ef..f6fd106fa 100644 --- a/rust-version +++ b/rust-version @@ -1 +1 @@ -4ecd70ddd1039a3954056c1071e40278048476fa +30f168ef811aec63124eac677e14699baa9395bd