Move out enum tests into a separate CXXInterop/enum directory

[scentini]

Currently, tests are scattered throughout multiple directories, making
it difficult to determine what is tested and what is not tested.
With upcoming work on interoperability between Swift and C++
(https://forums.swift.org/t/manifesto-interoperability-between-swift-and-c/33874),
we'd like to rearrange the tests and bring tests covering a single C/C++
feature next to each other.
This PR also adds tests for enabling C++ interop where possible.

[gribozavr]

@CodaFi Could you take a look? What do you think about this type of test reorganization?

As @scentini says, the tests for C interop are in a poor state:

• tests are scattered throughout multiple directories (ClangImporter, Sema, SILGen, IRGen etc.),
• various features are tested inconsistently,
• tests are often organized around how/when the feature was implemented, not around the aspect being tested,
• multiple tests often reuse huge headers, which are scary to change because the scope of their reuse is unclear,
• there is no easy way to find out what is tested and what isn't, etc.

We would like to reorganize existing C interop tests and run them in two modes, with and without C++ interop, to ensure that the C "subset" of C++ still works.

We will put tests under test/CXXInterop/<feature>, with further subdirectories if needed.

In this PR @scentini provides a preview the idea -- more PRs will follow from us if we agree that this is a good way to organize these tests.

[CodaFi]

I agree with the high-level goal of imposing more structure on the test suite, I just think that specifically having a CXXInterop high-level component in particular is not the right idea here.

/ClangImporter is ultimately the right place for the vast majority of this work. Now, /ClangImporter is also not exactly in the best state right now, and is very heavily tied to Objective-C support. I would be more than happy to see /ClangImporter grow divisions for the (now) three languages it supports. Just spitballing names, but:

/ClangImporter/ObjC
/ClangImporter/C
/ClangImporter/Cxx
/ClangImporter/Mixed

That would let you preserve a lot of the existing stuff (god modulemaps, half-built SDKs, quasi-overlays, etc.) and enforce enough of a division to be a marked improvement over the status quo. This does not address the problem of where to put e.g. codegen-related tests, but I suspect that will be the minority of tests that get written.

[gribozavr]

That would let you preserve a lot of the existing stuff (god modulemaps, half-built SDKs, quasi-overlays, etc.)

I would very much like to split up the huge modulemaps and headers that we have accumulated over the years. Changing those is quite difficult and risky, because the blast radius is very big -- you don't know where and how they are used. It is unclear what details in those headers are actually important and safe to change, because tests don't get tested themselves. If you change a header and tests continue passing you don't know whether it is still testing the same thing and the change was a no-op, or whether the test is now testing something trivial and therefore passing. So large shared headers only ever grow more stuff.

I think there should be a more curated set of shared things that is rooted in reality (e.g., SDK mocks that only have APIs found in real Apple SDKs, no CoreCooling fiction).

The small number of SILGen and IRGen tests for interop I think is a bug, not a feature. I think we need to increase our test coverage for interop in those areas. To facilitate that, to me it would make sense to share the headers across the tests for the importer, type checker, and code generation, and even execution tests. This way you know that a given C/Objective-C/C++ feature works across all stages of the compiler.

[CodaFi]

I think there should be a more curated set of shared things that is rooted in reality

Right, but /CXXInterop is not, in and of itself, going to address that and the other excellent test hygiene points you raised. And we should do these things regardless of what structure we settle on.

I would like to maintain the existing component-based divisions since that's not the actual source of friction from what you're describing. What I'm worried about here is that we're going to grow a /CXXInterop subsuite that has a hodge-podge of cross-cutting tests that have hygienic inputs and basically nothing else. A hygienic end-to-end integration test is pretty much impossible there - imagine a test that tries to conditionally import some C++ template-heavy code to test some diagnostics, conditionally re-import those tests to SILGen them, then imports them again to IRGen them. In order to impose order on that kind of subsuite, we'd need to return to a component-based hierarchy!

[gribozavr]

I would like to maintain the existing component-based divisions since that's not the actual source of friction from what you're describing.

I think the grouping of tests one of the reasons why the tests are not in a great state. In particular, because tests for various features are split across directories, we have inconsistent coverage.

A hygienic end-to-end integration test is pretty much impossible there - imagine a test that tries to conditionally import some C++ template-heavy code to test some diagnostics, conditionally re-import those tests to SILGen them, then imports them again to IRGen them. In order to impose order on that kind of subsuite, we'd need to return to a component-based hierarchy!

I'm not sure what you mean exactly, but what I was imagining was along the lines of:

CXXInterop/
   enum/
      Inputs/
        anonymous-enums.h
      anonymous-enums-typecheck.swift
      anonymous-enums-print-module-interface.swift
      anonymous-enums-silgen.swift
      anonymous-enums-irgen.sil
      anonymous-enums-execution.swift

anonymous-enums.h is used by tests right next to it (with obviously related filenames). These tests test the same feature, and therefore it is appropriate for them to reuse the same header -- but not for other tests.

I am suggesting that we continue to have tests that are testing individual compiler components as before, but grouping them around features, not around compiler components.

[CodaFi]

I'm not sure what you mean exactly, but what I was imagining was along the lines of:

Now multiply that discipline by 100. This directory is going to grow 300-500 feature-oriented tests files and the inputs directory is going to grow wild just like the others, it's just going to be atomized this time. If the goal is to more strongly correlate Inputs, then we can do it without /CXXInterop by properly encapsulating tests and their inputs under separate subdirectories, even feature-oriented ones as you propose.

ClangImporter/
   Cxx/
      anonymous-enums/
        anonymous-enums-typecheck.swift
        anonymous-enums-print-module-interface.swift
        anonymous-enums-silgen.swift
        anonymous-enums-irgen.sil
        anonymous-enums-execution.swift
        Inputs/
          anonymous-enums.h

I think a few tests try to do this, but most people are just chucking stuff into the nearby component's Inputs because it's convenient - SourceKit is a good model citizen in this regard. Just by counting RUN lines in files, /ClangImporter has 31 tests that emit SIL, 21 tests that emit IR, and 155 tests that typecheck stuff. It's fine to put your grouped end-to-end feature tests in there.

I think the grouping of tests one of the reasons why the tests are not in a great state

Maybe, but I think it's more that we have a richer story for semantic analysis that makes it more amenable to regression testing than the FileCheck-oriented stuff we do for codegen testing. We should optimize for that world. Unless the goal is to eventually redo the entire test suite like this with the proposed structure for /CXXInterop serving as a model.

[gribozavr]

I very much like scoping input files even more! The reason I was afraid to suggest that is that it felt like too big of a departure from today's test structure.

However... I think it is turning into a bigger discussion than we initially anticipated, and probably more people will have an opinion about this. So we wrote down our thoughts here on the forum: https://forums.swift.org/t/reorganize-swift-compiler-tests-for-c-objc-c-interop/34411 -- could you take a look?

[gribozavr]

@CodaFi Based on the discussion on the forums I suggest that we proceed.

[CodaFi]

Sorry for leaving this by the wayside. Agreed.

@swift-ci test and merge

[owenv]

I think this may have caused a regression on Windows, CXXInterop/enum/enum-anon-sized.swift is failing on recent runs: https://ci-external.swift.org/job/swift-PR-windows/1738/console