From ec9cd4692d83f7f6b3311d5de6ca8201c54067e1 Mon Sep 17 00:00:00 2001 From: Will Hawkins Date: Wed, 22 Nov 2023 10:12:04 -0500 Subject: [PATCH 01/11] [libcxx][docs] Make test name pattern documentation more obvious As a new contributor, I found it hard to find the documentation for the meaning of the names of different tests and how those names translate to Lit. This makes that documentation more explicit. --- libcxx/docs/TestingLibcxx.rst | 49 ++++++++++++++++++++++++++++++----- 1 file changed, 42 insertions(+), 7 deletions(-) diff --git a/libcxx/docs/TestingLibcxx.rst b/libcxx/docs/TestingLibcxx.rst index 44f3904f4e426..5783cc6a21922 100644 --- a/libcxx/docs/TestingLibcxx.rst +++ b/libcxx/docs/TestingLibcxx.rst @@ -330,13 +330,48 @@ additional headers. taken to make it work in earlier language versions. -Additional reading ------------------- - -The function ``CxxStandardLibraryTest`` in the file -``libcxx/utils/libcxx/test/format.py`` has documentation about writing test. It -explains the difference between the test named ``foo.pass.cpp`` and named -``foo.verify.cpp`` are. +Test names +---------- + +The names of test files have meaning for the libcxx-specific configuration of +Lit. A complete description can be found in the docstring for the function +``CxxStandardLibraryTest`` in the file ``libcxx/utils/libcxx/test/format.py``. + +For quick reference, here is an overview of some of the most useful patterns: + +.. list-table:: Lit Meaning of libcxx Test Filenames + :widths: 25 75 + :header-rows: 1 + + * - Name Pattern + - Meaning + * - ``FOO.pass.cpp`` + - Compiles, links and runs successfully + * - ``FOO.pass.mm`` + - Same as ``FOO.pass.cpp``, but for Objective-C++ + + * - ``FOO.compile.pass.cpp`` + - Compiles successfully, link and run not attempted + * - ``FOO.compile.pass.mm`` + - Same as ``FOO.compile.pass.cpp``, but for Objective-C++ + * - ``FOO.compile.fail.cpp`` + - Does not compile successfully + + * - ``FOO.link.pass.cpp`` + - Compiles and links successfully, run not attempted + * - ``FOO.link.pass.mm`` + - Same as ``FOO.link.pass.cpp``, but for Objective-C++ + * - ``FOO.link.fail.cpp`` + - Compiles successfully, but fails to link + + * - ``FOO.sh.`` + - A builtin Lit Shell test + + * - ``FOO.gen.`` + - A ``.sh`` test that generates one or more Lit tests on the fly. Executing this test must generate one or more files as expected by LLVM split-file, and each generated file leads to a separate Lit test that runs that file as defined by the test format. This can be used to generate multiple Lit tests from a single source file, which is useful for testing repetitive properties in the library. Be careful not to abuse this since this is not a replacement for usual code reuse techniques. + + * - ``FOO.verify.cpp`` + - Compiles with clang-verify. This type of test is automatically marked as UNSUPPORTED if the compiler does not support clang-verify. Benchmarks ========== From 11d5ec3ab6ebfe6ce1342adfa5c48e1221e353f7 Mon Sep 17 00:00:00 2001 From: Will Hawkins Date: Wed, 22 Nov 2023 21:21:02 -0500 Subject: [PATCH 02/11] fixup! [libcxx][docs] Make test name pattern documentation more obvious Update to consolidate lit test file name pattern semantic documentation to docs/ --- libcxx/utils/libcxx/test/format.py | 36 ++++-------------------------- 1 file changed, 4 insertions(+), 32 deletions(-) diff --git a/libcxx/utils/libcxx/test/format.py b/libcxx/utils/libcxx/test/format.py index e58e404bfcd2a..ace208c98459d 100644 --- a/libcxx/utils/libcxx/test/format.py +++ b/libcxx/utils/libcxx/test/format.py @@ -151,38 +151,10 @@ class CxxStandardLibraryTest(lit.formats.FileBasedTest): """ Lit test format for the C++ Standard Library conformance test suite. - This test format is based on top of the ShTest format -- it basically - creates a shell script performing the right operations (compile/link/run) - based on the extension of the test file it encounters. It supports files - with the following extensions: - - FOO.pass.cpp - Compiles, links and runs successfully - FOO.pass.mm - Same as .pass.cpp, but for Objective-C++ - - FOO.compile.pass.cpp - Compiles successfully, link and run not attempted - FOO.compile.pass.mm - Same as .compile.pass.cpp, but for Objective-C++ - FOO.compile.fail.cpp - Does not compile successfully - - FOO.link.pass.cpp - Compiles and links successfully, run not attempted - FOO.link.pass.mm - Same as .link.pass.cpp, but for Objective-C++ - FOO.link.fail.cpp - Compiles successfully, but fails to link - - FOO.sh. - A builtin Lit Shell test - - FOO.gen. - A .sh test that generates one or more Lit tests on the - fly. Executing this test must generate one or more files - as expected by LLVM split-file, and each generated file - leads to a separate Lit test that runs that file as - defined by the test format. This can be used to generate - multiple Lit tests from a single source file, which is - useful for testing repetitive properties in the library. - Be careful not to abuse this since this is not a replacement - for usual code reuse techniques. - - FOO.verify.cpp - Compiles with clang-verify. This type of test is - automatically marked as UNSUPPORTED if the compiler - does not support Clang-verify. - + Lit tests are contained in files that follow a certain pattern. That + pattern determines the semantics of the test. See + libcxx/docs/TestingLibcxx.rst or https://libcxx.llvm.org/TestingLibcxx.html + for a complete description of those semantics. Substitution requirements =============================== From a3a77408cd80c9de5512c078c83e285ce3e0ba85 Mon Sep 17 00:00:00 2001 From: Will Hawkins Date: Mon, 27 Nov 2023 09:15:47 -0500 Subject: [PATCH 03/11] fixup! [libcxx][docs] Make test name pattern documentation more obvious (Fully) centralize all naming documentation in the rst-based documentation. --- libcxx/docs/TestingLibcxx.rst | 41 ++++++++++++++++++++--------------- 1 file changed, 23 insertions(+), 18 deletions(-) diff --git a/libcxx/docs/TestingLibcxx.rst b/libcxx/docs/TestingLibcxx.rst index 5783cc6a21922..b93f61388cda6 100644 --- a/libcxx/docs/TestingLibcxx.rst +++ b/libcxx/docs/TestingLibcxx.rst @@ -334,11 +334,12 @@ Test names ---------- The names of test files have meaning for the libcxx-specific configuration of -Lit. A complete description can be found in the docstring for the function -``CxxStandardLibraryTest`` in the file ``libcxx/utils/libcxx/test/format.py``. - -For quick reference, here is an overview of some of the most useful patterns: +Lit. Based on the pattern that matches the name of a test file, Lit will test +the code contained therein in different ways. Refer to the `Lit Meaning of libcxx +Test Filenames`_ when +determining the names for new test files. +.. _Lit Meaning of libcxx Test Filenames: .. list-table:: Lit Meaning of libcxx Test Filenames :widths: 25 75 :header-rows: 1 @@ -346,32 +347,36 @@ For quick reference, here is an overview of some of the most useful patterns: * - Name Pattern - Meaning * - ``FOO.pass.cpp`` - - Compiles, links and runs successfully + - Checks whether the C++ code in the file compiles, links and runs successfully. * - ``FOO.pass.mm`` - - Same as ``FOO.pass.cpp``, but for Objective-C++ + - Same as ``FOO.pass.cpp``, but for Objective-C++. * - ``FOO.compile.pass.cpp`` - - Compiles successfully, link and run not attempted + - Checks whether the C++ code in the file compiles successfully. * - ``FOO.compile.pass.mm`` - - Same as ``FOO.compile.pass.cpp``, but for Objective-C++ + - Same as ``FOO.compile.pass.cpp``, but for Objective-C++. * - ``FOO.compile.fail.cpp`` - - Does not compile successfully + - Checks that the code in the file does *not* compile successfully. + + * - ``FOO.verify.cpp`` + - Compiles with clang-verify. This type of test is automatically marked as UNSUPPORTED if the compiler does not support clang-verify. * - ``FOO.link.pass.cpp`` - - Compiles and links successfully, run not attempted + - Checks that the C++ code in the file compiles and links successfully -- no run attempted. * - ``FOO.link.pass.mm`` - - Same as ``FOO.link.pass.cpp``, but for Objective-C++ + - Same as ``FOO.link.pass.cpp``, but for Objective-C++. * - ``FOO.link.fail.cpp`` - - Compiles successfully, but fails to link + - Checks whether the C++ code in the file fails to link after successful compilation. + * - ``FOO.link.fail.mm`` + - Same as ``FOO.link.fail.cpp``, but for Objective-C++. * - ``FOO.sh.`` - - A builtin Lit Shell test - + - A *builtin Lit Shell* test. * - ``FOO.gen.`` - - A ``.sh`` test that generates one or more Lit tests on the fly. Executing this test must generate one or more files as expected by LLVM split-file, and each generated file leads to a separate Lit test that runs that file as defined by the test format. This can be used to generate multiple Lit tests from a single source file, which is useful for testing repetitive properties in the library. Be careful not to abuse this since this is not a replacement for usual code reuse techniques. - - * - ``FOO.verify.cpp`` - - Compiles with clang-verify. This type of test is automatically marked as UNSUPPORTED if the compiler does not support clang-verify. + - A *builtin Lit Shell* test that generates one or more Lit tests on the fly. Executing this test must generate one or more files as expected + by LLVM split-file. Each generated file will drive an invocation of a separate Lit test. The format of the generated file will determine the type + of Lit test to be executed. This can be used to generate multiple Lit tests from a single source file, which is useful for testing repetitive properties + in the library. Be careful not to abuse this since this is not a replacement for usual code reuse techniques. Benchmarks ========== From d272cade10752310fd0fc18942cc4fe8efcd1aab Mon Sep 17 00:00:00 2001 From: Will Hawkins Date: Mon, 27 Nov 2023 09:19:29 -0500 Subject: [PATCH 04/11] fixup! [libcxx][docs] Make test name pattern documentation more obvious Point only to www documentation for test names (per @mordante's excellent feedback) --- libcxx/utils/libcxx/test/format.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libcxx/utils/libcxx/test/format.py b/libcxx/utils/libcxx/test/format.py index ace208c98459d..19577f84d24eb 100644 --- a/libcxx/utils/libcxx/test/format.py +++ b/libcxx/utils/libcxx/test/format.py @@ -153,7 +153,7 @@ class CxxStandardLibraryTest(lit.formats.FileBasedTest): Lit tests are contained in files that follow a certain pattern. That pattern determines the semantics of the test. See - libcxx/docs/TestingLibcxx.rst or https://libcxx.llvm.org/TestingLibcxx.html + https://libcxx.llvm.org/TestingLibcxx.html#test-names for a complete description of those semantics. Substitution requirements From 29fd8e3ee7b8c5dc12535c39bb8e7b99768d0ade Mon Sep 17 00:00:00 2001 From: Will Hawkins Date: Wed, 13 Dec 2023 22:12:52 -0500 Subject: [PATCH 05/11] Update libcxx/docs/TestingLibcxx.rst Be more clear about what type of shell we get when running these tests. Co-authored-by: Louis Dionne --- libcxx/docs/TestingLibcxx.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libcxx/docs/TestingLibcxx.rst b/libcxx/docs/TestingLibcxx.rst index b93f61388cda6..5ccac803f793c 100644 --- a/libcxx/docs/TestingLibcxx.rst +++ b/libcxx/docs/TestingLibcxx.rst @@ -373,7 +373,7 @@ determining the names for new test files. * - ``FOO.sh.`` - A *builtin Lit Shell* test. * - ``FOO.gen.`` - - A *builtin Lit Shell* test that generates one or more Lit tests on the fly. Executing this test must generate one or more files as expected + - A variant of a *Lit Shell* test that generates one or more Lit tests on the fly. Executing this test must generate one or more files as expected by LLVM split-file. Each generated file will drive an invocation of a separate Lit test. The format of the generated file will determine the type of Lit test to be executed. This can be used to generate multiple Lit tests from a single source file, which is useful for testing repetitive properties in the library. Be careful not to abuse this since this is not a replacement for usual code reuse techniques. From b538e9438e135c9ab5f787fe751b2e75f57c2b19 Mon Sep 17 00:00:00 2001 From: Will Hawkins Date: Wed, 13 Dec 2023 22:14:56 -0500 Subject: [PATCH 06/11] Update libcxx/utils/libcxx/test/format.py Make it clearer how the tests are generated by libcxx to drive lit according to the patterns embedded in the filenames that contain tests. Co-authored-by: Louis Dionne --- libcxx/utils/libcxx/test/format.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/libcxx/utils/libcxx/test/format.py b/libcxx/utils/libcxx/test/format.py index 19577f84d24eb..0a1f8716cd396 100644 --- a/libcxx/utils/libcxx/test/format.py +++ b/libcxx/utils/libcxx/test/format.py @@ -151,8 +151,9 @@ class CxxStandardLibraryTest(lit.formats.FileBasedTest): """ Lit test format for the C++ Standard Library conformance test suite. - Lit tests are contained in files that follow a certain pattern. That - pattern determines the semantics of the test. See + Lit tests are contained in files that follow a certain pattern, which determines the semantics of the test. + Under the hood, we basically generate a builtin Lit shell test that follows the ShTest format, and perform + the appropriate operations (compile/link/run). See https://libcxx.llvm.org/TestingLibcxx.html#test-names for a complete description of those semantics. From 7efd1ec22fb08ed15b6eb2f9fde2d549cf40118c Mon Sep 17 00:00:00 2001 From: Will Hawkins Date: Wed, 13 Dec 2023 22:59:59 -0500 Subject: [PATCH 07/11] fixup! [libcxx][docs] Make test name pattern documentation more obvious Add documentation about ADDITIONAL_COMPILE_FLAGS and FILE_DEPENDENCIES. --- libcxx/docs/TestingLibcxx.rst | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/libcxx/docs/TestingLibcxx.rst b/libcxx/docs/TestingLibcxx.rst index 5ccac803f793c..1b956434cefa6 100644 --- a/libcxx/docs/TestingLibcxx.rst +++ b/libcxx/docs/TestingLibcxx.rst @@ -378,6 +378,36 @@ determining the names for new test files. of Lit test to be executed. This can be used to generate multiple Lit tests from a single source file, which is useful for testing repetitive properties in the library. Be careful not to abuse this since this is not a replacement for usual code reuse techniques. + +libc++-Specific Lit Features +---------------------------- + +Custom Directives +~~~~~~~~~~~~~~~~~ + +Lit has many directives built in (e.g., ``DEFINE``, ``UNSUPPORTED``). In addition to those directives, libcxx adds two additional libc++-specific directives that makes +writing tests easier. See `libc++-specific Lit Directives`_ for more information about the ``FILE_DEPENDENCIES`` and ``ADDITIONAL_COMPILE_FLAGS`` libc++-specific directives. + +.. _libc++-specific Lit Directives: +.. list-table:: libc++-specific Lit Directives + :widths: 20 35 45 + :header-rows: 1 + + * - Directive + - Parameters + - Usage + * - ``FILE_DEPENDENCIES`` + - ``// FILE_DEPENDENCIES: file, directory, /path/to/file, ...`` + - The paths given to the ``FILE_DEPENDENCIES`` directive can specify directories or specific files upon which a given test depend. Copies of the files individually specified and + *all* the files in any specified directories will be placed in the *current working directory* of the test when it executes. All dependency directories and files + specified using relative paths will be anchored to the directory specified by the ``%S`` substitution (i.e., the source directory of the test being executed.). + * - ``ADDITIONAL_COMPILE_FLAGS`` + - ``// ADDITIONAL_COMPILE_FLAGS: flag1, flag2, ...`` + - The additional compiler flags specified using the ``ADDITIONAL_COMPILE_FLAGS`` libc++-specific Lit directive will be added to the invocation of ``clang`` used to build + the test executables. ``ADDITIONAL_COMPILE_FLAGS`` are only used with tests that must be built from source using clang. In other words, ``FOO.sh.`` tests will not use + the value of this directive. The clang compiler flags specified in the ``ADDITIONAL_COMPILE_FLAGS`` libc++-specific Lit directive are added *after* the compiler flags + specified in the ``%{compile_flags}`` substitution. + Benchmarks ========== From 55b8a077f768d017c2cb2be63e1d19a642c989ba Mon Sep 17 00:00:00 2001 From: Will Hawkins Date: Thu, 21 Dec 2023 15:57:19 -0500 Subject: [PATCH 08/11] fixup! [libcxx][docs] Make test name pattern documentation more obvious Address @ldionne's astute feedback. --- libcxx/docs/TestingLibcxx.rst | 33 +++++++++++++++--------------- libcxx/utils/libcxx/test/format.py | 24 ---------------------- 2 files changed, 17 insertions(+), 40 deletions(-) diff --git a/libcxx/docs/TestingLibcxx.rst b/libcxx/docs/TestingLibcxx.rst index 1b956434cefa6..3e9dab0581400 100644 --- a/libcxx/docs/TestingLibcxx.rst +++ b/libcxx/docs/TestingLibcxx.rst @@ -325,7 +325,7 @@ This macro is in a different header as ``assert_macros.h`` since it pulls in additional headers. .. note: This macro can only be used in test using C++20 or newer. The macro - was added at a time where most of lib++'s C++17 support was complete. + was added at a time where most of libc++'s C++17 support was complete. Since it is not expected to add this to existing tests no effort was taken to make it work in earlier language versions. @@ -333,14 +333,13 @@ additional headers. Test names ---------- -The names of test files have meaning for the libcxx-specific configuration of +The names of test files have meaning for the libc++-specific configuration of Lit. Based on the pattern that matches the name of a test file, Lit will test -the code contained therein in different ways. Refer to the `Lit Meaning of libcxx -Test Filenames`_ when -determining the names for new test files. +the code contained therein in different ways. Refer to the `Lit Meaning of libc++ +Test Filenames`_ when determining the names for new test files. -.. _Lit Meaning of libcxx Test Filenames: -.. list-table:: Lit Meaning of libcxx Test Filenames +.. _Lit Meaning of libc++ Test Filenames: +.. list-table:: Lit Meaning of libc++ Test Filenames :widths: 25 75 :header-rows: 1 @@ -385,7 +384,7 @@ libc++-Specific Lit Features Custom Directives ~~~~~~~~~~~~~~~~~ -Lit has many directives built in (e.g., ``DEFINE``, ``UNSUPPORTED``). In addition to those directives, libcxx adds two additional libc++-specific directives that makes +Lit has many directives built in (e.g., ``DEFINE``, ``UNSUPPORTED``). In addition to those directives, libc++ adds two additional libc++-specific directives that makes writing tests easier. See `libc++-specific Lit Directives`_ for more information about the ``FILE_DEPENDENCIES`` and ``ADDITIONAL_COMPILE_FLAGS`` libc++-specific directives. .. _libc++-specific Lit Directives: @@ -398,15 +397,17 @@ writing tests easier. See `libc++-specific Lit Directives`_ for more information - Usage * - ``FILE_DEPENDENCIES`` - ``// FILE_DEPENDENCIES: file, directory, /path/to/file, ...`` - - The paths given to the ``FILE_DEPENDENCIES`` directive can specify directories or specific files upon which a given test depend. Copies of the files individually specified and - *all* the files in any specified directories will be placed in the *current working directory* of the test when it executes. All dependency directories and files - specified using relative paths will be anchored to the directory specified by the ``%S`` substitution (i.e., the source directory of the test being executed.). + - The paths given to the ``FILE_DEPENDENCIES`` directive can specify directories or specific files upon which a given test depend. For example, a test that requires some test + input stored in a data file would use this libc++-specific Lit directive. When a test file contains the ``FILE_DEPENDENCIES`` directive, Lit will collect the named files and copy + them to the directory represented by the ``%T`` substitution before the test executes. The copy is performed from the directory represented by the ``%S`` substitution + (i.e. the source directory of the test being executed) which makes it possible to use relative paths to specify the location of dependency files. After Lit copies + all the dependent files to the directory specified by the ``%T`` substitution, that directory should contain *all* the necessary inputs to run. In other words, + it should be possible to copy the contents of the directory specified by the ``%T`` substitution to a remote host where the execution of the test will actually occur. * - ``ADDITIONAL_COMPILE_FLAGS`` - - ``// ADDITIONAL_COMPILE_FLAGS: flag1, flag2, ...`` - - The additional compiler flags specified using the ``ADDITIONAL_COMPILE_FLAGS`` libc++-specific Lit directive will be added to the invocation of ``clang`` used to build - the test executables. ``ADDITIONAL_COMPILE_FLAGS`` are only used with tests that must be built from source using clang. In other words, ``FOO.sh.`` tests will not use - the value of this directive. The clang compiler flags specified in the ``ADDITIONAL_COMPILE_FLAGS`` libc++-specific Lit directive are added *after* the compiler flags - specified in the ``%{compile_flags}`` substitution. + - ``// ADDITIONAL_COMPILE_FLAGS: flag1 flag2 ...`` + - The additional compiler flags specified by a space-separated list to the ``ADDITIONAL_COMPILE_FLAGS`` libc++-specific Lit directive will be added to the end of the ``%{compile_flags}`` + substitution for the test that contains it. This libc++-specific Lit directive makes it possible to add special compilation flags without having to resort to writing a ``.sh.cpp`` test (see + `Lit Meaning of libc++ Test Filenames`_), more powerful but perhaps overkill. Benchmarks ========== diff --git a/libcxx/utils/libcxx/test/format.py b/libcxx/utils/libcxx/test/format.py index 0a1f8716cd396..5d84711bf5d28 100644 --- a/libcxx/utils/libcxx/test/format.py +++ b/libcxx/utils/libcxx/test/format.py @@ -173,30 +173,6 @@ class CxxStandardLibraryTest(lit.formats.FileBasedTest): in the same command line. In other words, the test format doesn't perform separate compilation and linking steps in this case. - - Additional supported directives - =============================== - In addition to everything that's supported in Lit ShTests, this test format - also understands the following directives inside test files: - - // FILE_DEPENDENCIES: file, directory, /path/to/file - - This directive expresses that the test requires the provided files - or directories in order to run. An example is a test that requires - some test input stored in a data file. When a test file contains - such a directive, this test format will collect them and copy them - to the directory represented by %T. The intent is that %T contains - all the inputs necessary to run the test, such that e.g. execution - on a remote host can be done by simply copying %T to the host. - - // ADDITIONAL_COMPILE_FLAGS: flag1 flag2 flag3 - - This directive will cause the provided flags to be added to the - %{compile_flags} substitution for the test that contains it. This - allows adding special compilation flags without having to use a - .sh.cpp test, which would be more powerful but perhaps overkill. - - Additional provided substitutions and features ============================================== The test format will define the following substitutions for use inside tests: From be73e31b168d34c6102bc0ecdfc5995517a5c64e Mon Sep 17 00:00:00 2001 From: Will Hawkins Date: Tue, 26 Dec 2023 11:36:21 -0500 Subject: [PATCH 09/11] fixup! [libcxx][docs] Make test name pattern documentation more obvious Add link to Internals Manual for writing clang-verify tests. --- libcxx/docs/TestingLibcxx.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/libcxx/docs/TestingLibcxx.rst b/libcxx/docs/TestingLibcxx.rst index 3e9dab0581400..4f27dd1f5ad24 100644 --- a/libcxx/docs/TestingLibcxx.rst +++ b/libcxx/docs/TestingLibcxx.rst @@ -359,6 +359,7 @@ Test Filenames`_ when determining the names for new test files. * - ``FOO.verify.cpp`` - Compiles with clang-verify. This type of test is automatically marked as UNSUPPORTED if the compiler does not support clang-verify. + For additional information about how to write tests using ``clang-verify``, see the `Internals Manual `_. * - ``FOO.link.pass.cpp`` - Checks that the C++ code in the file compiles and links successfully -- no run attempted. From a5f8fbde3e719d22a9fe12fb93375854c7b0202a Mon Sep 17 00:00:00 2001 From: Will Hawkins Date: Fri, 29 Dec 2023 20:16:38 -0500 Subject: [PATCH 10/11] fixup! [libcxx][docs] Make test name pattern documentation more obvious Add recommendation on when to write verify tests. --- libcxx/docs/TestingLibcxx.rst | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/libcxx/docs/TestingLibcxx.rst b/libcxx/docs/TestingLibcxx.rst index 4f27dd1f5ad24..2a4a26c493d67 100644 --- a/libcxx/docs/TestingLibcxx.rst +++ b/libcxx/docs/TestingLibcxx.rst @@ -351,7 +351,8 @@ Test Filenames`_ when determining the names for new test files. - Same as ``FOO.pass.cpp``, but for Objective-C++. * - ``FOO.compile.pass.cpp`` - - Checks whether the C++ code in the file compiles successfully. + - Checks whether the C++ code in the file compiles successfully. In general, prefer `compile` tests over `verify` tests, + subject to the specific recommendations, below, for when to write `verify` tests. * - ``FOO.compile.pass.mm`` - Same as ``FOO.compile.pass.cpp``, but for Objective-C++. * - ``FOO.compile.fail.cpp`` @@ -359,7 +360,14 @@ Test Filenames`_ when determining the names for new test files. * - ``FOO.verify.cpp`` - Compiles with clang-verify. This type of test is automatically marked as UNSUPPORTED if the compiler does not support clang-verify. - For additional information about how to write tests using ``clang-verify``, see the `Internals Manual `_. + For additional information about how to write `verify` tests, see the `Internals Manual `_. + Prefer `verify` tests over `compile` tests to test that compilation fails for a particular reason. For example, use a `verify` test + to ensure that + + * an expected ``static_assert`` is triggered; + * the use of deprecated functions generates the proper warning; + * removed functions are no longer usable; or + * return values from functions marked ``[[nodiscard]]`` are stored. * - ``FOO.link.pass.cpp`` - Checks that the C++ code in the file compiles and links successfully -- no run attempted. From 4f0e726967a1d57a1eae568a0ffcaaf731691841 Mon Sep 17 00:00:00 2001 From: Will Hawkins Date: Mon, 8 Jan 2024 16:54:03 -0500 Subject: [PATCH 11/11] fixup! [libcxx][docs] Make test name pattern documentation more obvious Address final comments. --- libcxx/docs/TestingLibcxx.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/libcxx/docs/TestingLibcxx.rst b/libcxx/docs/TestingLibcxx.rst index 2a4a26c493d67..e7645cb5885f1 100644 --- a/libcxx/docs/TestingLibcxx.rst +++ b/libcxx/docs/TestingLibcxx.rst @@ -351,8 +351,8 @@ Test Filenames`_ when determining the names for new test files. - Same as ``FOO.pass.cpp``, but for Objective-C++. * - ``FOO.compile.pass.cpp`` - - Checks whether the C++ code in the file compiles successfully. In general, prefer `compile` tests over `verify` tests, - subject to the specific recommendations, below, for when to write `verify` tests. + - Checks whether the C++ code in the file compiles successfully. In general, prefer ``compile`` tests over ``verify`` tests, + subject to the specific recommendations, below, for when to write ``verify`` tests. * - ``FOO.compile.pass.mm`` - Same as ``FOO.compile.pass.cpp``, but for Objective-C++. * - ``FOO.compile.fail.cpp`` @@ -360,8 +360,8 @@ Test Filenames`_ when determining the names for new test files. * - ``FOO.verify.cpp`` - Compiles with clang-verify. This type of test is automatically marked as UNSUPPORTED if the compiler does not support clang-verify. - For additional information about how to write `verify` tests, see the `Internals Manual `_. - Prefer `verify` tests over `compile` tests to test that compilation fails for a particular reason. For example, use a `verify` test + For additional information about how to write ``verify`` tests, see the `Internals Manual `_. + Prefer `verify` tests over ``compile`` tests to test that compilation fails for a particular reason. For example, use a ``verify`` test to ensure that * an expected ``static_assert`` is triggered;