Skip to content

[docs] GettingStarted: Default to "hosttools" bootstrapping mode on macOS #71013

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

AnthonyLatsis
Copy link
Collaborator

@AnthonyLatsis AnthonyLatsis commented Jan 19, 2024

To substantiate the bootstrapping mode:

  • Incremental builds with bootstrapping-with-hostlibs are painfully and unnecessarily slow for newcomers and most maintainers.
  • bootstrapping-with-hostlibs is the macOS default that is also used in CI. However, there are surely far more active contributors using hosttools, so this should be pretty safe.
  • My understanding is that hosttools is the default we are aiming for. We’re gonna have to support building Swift in Swift with a stable release at some point.

@AnthonyLatsis AnthonyLatsis force-pushed the getting-started-hosttools-bootstrapping branch from 9eb0c5f to 77dc017 Compare January 19, 2024 17:39
@AnthonyLatsis AnthonyLatsis requested a review from hborla January 19, 2024 17:40
@AnthonyLatsis AnthonyLatsis force-pushed the getting-started-hosttools-bootstrapping branch from 77dc017 to 17acfb3 Compare January 19, 2024 17:52
@AnthonyLatsis AnthonyLatsis force-pushed the getting-started-hosttools-bootstrapping branch from 17acfb3 to 7996de1 Compare January 19, 2024 17:55
@egorzhdan egorzhdan requested a review from eeckstein January 21, 2024 01:09
> **Note**
> If you are planning to work on the compiler on macOS, but not the parts that are
> written in Swift, pass `--bootstrapping=hosttools` to speed up local
> development.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice if the converse of this was kept, ie telling people working on the Swift parts what to do.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The converse is simply the default (not to supply the mode). But I suspect very few people actually use 2-stage bootstrapping for development because the editing cycle is a lot slower. Personally, I would rather rely on CI to catch any instances of a just-built compiler not being happy with whatever I did using my host compiler.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should make hosttools the default mode

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sneaky! That’s even better. macOS default is bootstrapping-with-hostlibs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The note you are deleting says the opposite, that hosttools helps with writing the C++ portions, which I don't know why though.

Copy link
Collaborator Author

@AnthonyLatsis AnthonyLatsis Jan 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because in hosttools mode the Swift modules are not rebuilt from scratch to bootstrap the compiler in response to a change in a C++ file. Suggesting hosttools in both cases makes the note apply to writing Swift parts too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I see, hosttools works well for both, got it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the confusion. Still learning better ways to express myself.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, you said that above: I just wasn't sure that's what you meant until you went into detail about avoiding the full bootstrap. 😄

If the build fails, see [Troubleshooting build issues](#troubleshooting-build-issues).
- If you installed and want to use Sccache, add `--sccache` to the invocation.
- If you want to use a debugger such as LLDB on compiler sources, add
`--debug-swift` to the invocation.
Copy link
Member

@finagolfin finagolfin Jan 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you want to replace --release-debuginfo with --debug-swift, not just add it. Also, the previous note explained why, might be good to keep some of that.

Copy link
Collaborator Author

@AnthonyLatsis AnthonyLatsis Jan 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To my knowledge, replacing it is equivalent to not specifying --debug-swift at all, since the default overall build variant is --debug. My experience as a contributor tells me that a newcomer that actually needs a full debug build is an exception. I am yet to feel a need of e.g. stepping over LLVM code when working on the compiler myself.

Also, the previous note explained why, might be good to keep some of that

👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm looking into what is set on linux with the different commands, will let you know what I find.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right: --release-debuginfo sets that for all repos, while --debug-swift successfully overrides that for just the Swift compiler frontend, so you would use both. I was worried that they would not interact well, but they do.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also in my experience I've actually always specified --release and --debug-swift because I am on a machine with 128GB disk space only so, couldn't have that cost of a --release-debuginfo which is not needed for most of contributors working only on the swift target (although may be useful when you see a crash in an LLVM method for example). @AnthonyLatsis, although this topic is out of scope for this PR do you think is worth mentioning that somewhere here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LucianoPAlmeida Somehow I missed this, sorry. I do think that all these shenanigans that help you optimize build times and save disk space deserve a detailed discussion, but in a more dedicated place like the DevelopmentTips document. Seeing as we are moving toward replacing this content with a ready-to-go first-time-contributor-tuned preset and keeping things really simple, we have to make a choice. LLVM assertion failures are a common kind of crash and effect of playing around with unfamiliar code, and beginners make lots of mistakes, so I think the debug info is generally a justified asset for the aspiring contributor.

We absolutely should pursue other ways of improving this axis of experience, though. For example, optimize the checkout, skip non-essential targets, and provide more fine-grained configuration options.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We absolutely should pursue other ways of improving this axis of experience, though. For example, optimize the checkout, skip non-essential targets, and provide more fine-grained configuration options.

Sounds good @AnthonyLatsis :)
I think my point about optimize build times and save disk space is that it is important specially for people who want to contribute but are not on high-end machines, so that helps the experience overall for them.

@bnbarham
Copy link
Contributor

bootstrapping-with-hostlibs is the macOS default that is also used in CI. However, there are surely far more active contributors using hosttools, so this should be pretty safe.

FWIW I have #70343 up that is only pending some further checks. So the default will soon be hosttools.

@AnthonyLatsis AnthonyLatsis force-pushed the getting-started-hosttools-bootstrapping branch from 7996de1 to e7b8430 Compare January 23, 2024 23:58
@AnthonyLatsis AnthonyLatsis force-pushed the getting-started-hosttools-bootstrapping branch from e7b8430 to cb988a4 Compare January 25, 2024 22:39
@AnthonyLatsis
Copy link
Collaborator Author

@xedin ping

@xedin
Copy link
Contributor

xedin commented Feb 6, 2024

This is in my queue for this week, should be able to take a look soon!

@xedin
Copy link
Contributor

xedin commented Feb 10, 2024

I didn't forget but didn't have time to take a look, this is going to be the first thing on Monday!

@AnthonyLatsis
Copy link
Collaborator Author

Whenever you’re ready!

@AnthonyLatsis
Copy link
Collaborator Author

@swift-ci please smoke test

@AnthonyLatsis AnthonyLatsis merged commit bc132da into swiftlang:main Feb 12, 2024
@AnthonyLatsis AnthonyLatsis deleted the getting-started-hosttools-bootstrapping branch February 13, 2024 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants