diff --git a/tests/compiletests/README.md b/tests/compiletests/README.md index 1a3abc95b3..599f3b2899 100644 --- a/tests/compiletests/README.md +++ b/tests/compiletests/README.md @@ -1,23 +1,80 @@ # Compiletests -This folder contains tests known as "compiletests". Each file in the `ui` folder corresponds to a -single compiletest. The way they work is a tool iterates over every file, and tries to compile it. -At the start of the file, there's some meta-comments about the expected result of the compile: -whether it should succeed compilation, or fail. If it is expected to fail, there's a corresponding -.stderr file next to the file that contains the expected compiler error message. +This folder contains tests known as "compiletests", where each file in the `ui` folder corresponds +to a single compiletest. Keep in mind that the tests here are not executed, they are merely checked +for compile errors and the resulting binaries being valid (with spirv-val). If you also want to run +your shaders and verify their output, have a look at the neighbouring "difftests" instead. -The `src` folder here is the tool that iterates over every file in the `ui` folder. It uses the -`compiletests` library, taken from rustc's own compiletest framework. - -You can run compiletests via `cargo compiletest`. This is an alias set up in `.cargo/config` for +You can run compiletests via `cargo compiletest`, with an alias setup in `.cargo/config` for `cargo run --release -p compiletests --`. You can filter to run specific tests by passing the -(partial) filenames to `cargo compiletest some_file_name`, and update the `.stderr` files to -contain new output via the `--bless` flag (with `--bless`, make sure you're actually supposed to be -changing the .stderr files due to an intentional change, and hand-validate the output is correct -afterwards). - -Keep in mind that tests here here are not executed, merely checked for errors (including validating -the resulting binary with spirv-val). Because of this, there might be some strange code in here - -the point isn't to make a fully functional shader every time (that would take an annoying amount of -effort), but rather validate that specific parts of the compiler are doing their job correctly -(either succeeding as they should, or erroring as they should). +(partial) filenames like `cargo compiletest arch/subgroup`. You can also `--bless` the tests, +which will update their expected stderr output in their associated `.stderr` file, if you +promise to **manually verify** the new contents before committing them. + +Our compiletests use the [`compiletest_rs`](https://github.com/Manishearth/compiletest-rs) library, +which is the compiletest framework within rustc itself, and some glue code in `src`. + +## How to write a compiletest + +Create a new `test.rs` file within `./ui` and copy in the following minimal contents: +```rust +// build-pass + +use spirv_std::spirv; + +#[spirv(fragment)] +pub fn main() {} +``` + +The first lines are meta-comments containing instructions on how this test should be compiled, at +the very least `build-pass` or `build-fail`. Additional flags can be found in the next chapter. +Then you declare your shader entry point as you would normally, here a simple fragment shader that +does nothing. You can only use external `spirv_std` crate as well as it's reexported dependents, so +to get a `Vec4` you can `use spirv_std::glam::Vec4` and if you need complex f32 operations you can +`use spirv_std::num_traits::Float`, just like you would in a normal shader. + +The stderr output of the build operation is expected to match the contents of the file +`path/to/test.rs.stderr` and may be updated with `--bless`. The stderr output may either come from +some errors emitted by rustc or the rust-gpu itself, usually on tests which are expected to fail, +or from other flags to disassemble the shader. Please manually verify any changes to the output +before committing them. + +## Meta-comments reference + +Most meta-comments can be added multiple times, for example multiple `compile-flags` will be +appended to each other. Listed here are some of the common ones seen in rust-gpu compile tests, a +full reference can be found in the +[Compiletest documentation of rustc](https://rustc-dev-guide.rust-lang.org/tests/ui.html#error-annotations). + +* `// build-pass` the shader must build successfully +* `// build-fail` the shader must fail to build +* `// only-vulkan1.2` only run the test on target vulkan1.2, skip otherwise. Other targets are accepted. +* `// ignore-vulkan1.2` skip the test on target vulkan1.2. Other targets are accepted. +* `// compile-flags: something` adds `something` to the compile flags +* `// compile-flags: -C target-feature=+StorageImageExtendedFormats` add a `Capability` like with `SpirvBuilder.capabilities` +* `// compile-flags: -C +ext:SPV_KHR_ray_tracing` add an `Extension` like with `SpirvBuilder.extensions` +* `// compile-flags: -C target-feature=+MeshShadingEXT,+ext:SPV_EXT_mesh_shader` add both an extension and a capability, as they are often go together +* `// compile-flags: -C llvm-args=--abort-strategy=debug-printf` an example how to set some other property, see source of `SpirvBuilder` for reference + +### Disassembly + +All disassembly will be dumped to stderr, which makes the testing framework compare it to the +contents of `path/to/test.rs.stdout`. + +* `// compile-flags: -C llvm-args=--disassemble` disassemble the entire shader +* `// compile-flags: -C llvm-args=--disassemble-globals` disassemble only globals and function declarations, excludes function contents +* `// compile-flags: -C llvm-args=--disassemble-entry=main` disassembles the `main` entry point, does not work on functions +* `// compile-flags: -C llvm-args=--disassemble-fn=add_two_ints::add_two_ints` disassembles the + function `add_two_ints`. The name must be `::`, the function must + **not** be an entry point but must called by an entry point to not be dead-code eliminated. + Usually, a test has the disassembled function and an entry point calling it. +* `// normalize-stderr-test "targetSpecificMsg" -> ""` Replaces any substrings in stderr with another to normalise output + between different machines and targets. By default, you should have to not specify any and only add them as needed. + List of common substitutions: + * `// normalize-stderr-test "OpLine .*\n" -> ""` remove all line numbers from debug info + * `// normalize-stderr-test "OpSource .*\n" -> ""` remove all source code from debug info + * `// normalize-stderr-test "\S*/crates/spirv-std/src/" -> "$$SPIRV_STD_SRC/"` normalize path to the `spirv-std` crate + * `// normalize-stderr-test "OpCapability VulkanMemoryModel\n" -> ""` remove the vulkan memory model *capability*, only used by vulkan targets + * `// normalize-stderr-test "OpExtension .SPV_KHR_vulkan_memory_model.\n" -> ""` remove the vulkan memory model *extension*, only used by vulkan targets + * `// normalize-stderr-test "OpMemoryModel Logical Vulkan" -> "OpMemoryModel Logical Simple"` replace the `Vulkan` memory model with `Simple` + * `// normalize-stderr-test "\S*/lib/rustlib/" -> "$$SYSROOT/lib/rustlib/"` normalize path to crates delivered with rustc, such as `core` diff --git a/tests/difftests/tests/simple-compute/simple-compute-rust/Cargo.toml b/tests/difftests/tests/simple-compute/simple-compute-rust/Cargo.toml index b619027e6e..ab34115943 100644 --- a/tests/difftests/tests/simple-compute/simple-compute-rust/Cargo.toml +++ b/tests/difftests/tests/simple-compute/simple-compute-rust/Cargo.toml @@ -12,7 +12,6 @@ crate-type = ["dylib"] [dependencies] # GPU deps -[target.'cfg(target_arch = "spirv")'.dependencies] spirv-std.workspace = true # CPU deps diff --git a/tests/difftests/tests/simple-compute/simple-compute-rust/src/lib.rs b/tests/difftests/tests/simple-compute/simple-compute-rust/src/lib.rs index a398192b92..03be51b555 100644 --- a/tests/difftests/tests/simple-compute/simple-compute-rust/src/lib.rs +++ b/tests/difftests/tests/simple-compute/simple-compute-rust/src/lib.rs @@ -1,4 +1,3 @@ -#![cfg(target_arch = "spirv")] #![no_std] use spirv_std::spirv;