[embedded] Start building and including lld even in Darwin toolchains

[kubamracek]

We already include lld in the Linux and Windows toolchains. For embedded Swift development, it's common to (need to) use ELF even on a macOS host, but the Xcode-provided linker is Mach-O only. Let's start including a linker capable of linking ELF binaries (for embedded cross-compilation use cases) even in the Darwin toolchains.

[kubamracek]

@swift-ci please test

[MaxDesiatov]

Yes!

[kubamracek]

@swift-ci Please Build Toolchain macOS Platform

[kubamracek]

@swift-ci Please Build Toolchain macOS Platform

[kubamracek]

@shahmishal This will have Darwin toolchain size impact (should be no change on Linux/Windows toolchains).

[kubamracek]

@swift-ci please test

[shahmishal]

@swift-ci build toolchain macOS

[shahmishal]

I don't see lld getting installed in the toolchain, not sure if we are missing something.

[shahmishal]

We will need to change this to include lld

https://github.com/apple/swift/blob/main/utils/build-presets.ini#L22

[shahmishal]

@kubamracek I am a change to your PR to see if this fixes the issue.

[shahmishal]

@swift-ci build toolchain macOS

[kubamracek]

@swift-ci please test

[ole]

@kubamracek If you have the time, could you start another macOS toolchain build for me? Or do you have an idea when this PR will be merged? I have a feeling it's the only remaining blocker to have a working SwiftPM-based build for our RP2040 experiment, and I'd love to try this out.

It seems the last Mac toolchain build is no longer available? Because the CI server only keeps those around for a limited time? (I have no idea how this works.)

Thanks!

[kubamracek]

@swift-ci build toolchain macOS

[kubamracek]

@kubamracek If you have the time, could you start another macOS toolchain build for me? Or do you have an idea when this PR will be merged? I have a feeling it's the only remaining blocker to have a working SwiftPM-based build for our RP2040 experiment, and I'd love to try this out.

It seems the last Mac toolchain build is no longer available? Because the CI server only keeps those around for a limited time? (I have no idea how this works.)

Kicked off another toolchain build. You're right, the CI server only keeps a limited number of build logs and build artifacts.

I still need to figure out a few things about the inclusion of lld, there might be a bit more delays. I wonder, if all you need is a working lld on macOS, then if a possible short-term workaround for you would be to grab lld from llvm.org? The releases of LLVM for macOS do include it.

[ole]

I wonder, if all you need is a working lld on macOS, then if a possible short-term workaround for you would be to grab lld from llvm.org? The releases of LLVM for macOS do include it.

Thank you! Thanks for the llvm.org tip. I've been using the Clang from the ARM embedded LLVM toolchain, and that seems to work too, but it's obviously a little inconvenient for users.

Here's roughly my SwiftPM setup (not public yet, I have to write it all up):

1. One library target/product for the RP2040 boot2 bootloader. This is a bunch of assembly files, copied from the Pico SDK.

2. One library target for the RP2040 crt0. This is also some assembly files, plus one or two C files, copied from the Pico SDK.

3. A SwiftPM static library target/product for the actual RP2040 executable, written in Swift. This target depends on (2), but not on (1) because we need to postprocess the bootloader before linking (see below).

   Perhaps this should be an executable, but I don't think SwiftPM can build embedded Swift executables yet (I forget the exact error message I get when I try that)? And even if we can build an executable in this step, it wouldn't be enough because we need to inject our custom linker scripts etc.

4. A SwiftPM command plugin that does the following:

   • Build products (1) and (3) separately.
   • Postprocess (1), the bootloader: compute a checksum over the assembled binary, write the checksum into the binary, link everything to the correct location.
   • Call Clang to link the postprocessed bootloader together with (3) into an executable.

I have this working on Linux with the default Swift Docker images, so you can write something like this to build your executable:

swift package --triple armv6m-none-none-eabi link-executable

On macOS, it also works with a current toolchain as long as you have the ld.lld from the ARM embedded LLVM toolchain in your PATH.

[kubamracek]

Very cool, please do write it up, or even post this as is to the forums :) There are folks (like me) who are eager to read about doing cool stuff with Embedded Swift.

[MaxDesiatov]

Thank you! Thanks for the llvm.org tip. I've been using the Clang from the ARM embedded LLVM toolchain, and that seems to work too, but it's obviously a little inconvenient for users.

I found lld that comes with a plain brew install llvm is sufficient when cross-compiling from macOS.

[ole]

@MaxDesiatov Thank you, I'll try that.

[MaxDesiatov]

Note that the Homebrew install doesn't add it to the usual PATH locations, on Apple Silicon it's /opt/homebrew/opt/llvm/bin/ld.lld and on Intel it should be different from that.

[kubamracek]

@swift-ci please test

[MaxDesiatov]

Any chance this would be cherry-picked for 6.0?