extend-payload-to-esp: extends any src_input_dir to esp

[say-paul]

should fix: #651
To support raspberry pi 4, which works well for rpm-ostree, but in bootc. I intend(commented out) to copy over the uboot binary

RUN dnf install -y uboot-images-armv8
 # cp -P /usr/share/uboot/rpi_arm64/u-boot.bin /boot/efi/rpi-u-boot.bin

my steps to achieve this:

• If I have /usr/lib/uboot/rpi4/Uboot.bin
• When I do bootupd extend-payload-to-esp /usr/lib/uboot/rpi4
• Then content of /usr/lib/uboot/rpi4/ will be copied to /usr/lib/efi/firmware/<name>/<version>/EFI/boot/efi as per Extend generate-update-metadata to read from /usr (Unification of boot loader updates, phase 1) #926
  --This should stage it for Extend generate-update-metadata() to read from /usr #938 to move things to bootupd/update/ through generate-update-matadata
  --Future--
• Again when I do bootupd install
• Then content of /usr/lib/bootupd/updates/efi/EFI/ will be copied to /boot/esp/
• Finally We have /boot/efi/uboot.bin

[say-paul]

I am still trying to build an image with this feature , but with no luck,
My Containerfile looks like this:

FROM quay.io/fedora/fedora-iot:42-x86_64
RUN dnf install -y make cargo rust glib2-devel openssl-devel ostree-devel git
    
RUN git clone -b firmware-copy https://github.com/say-paul/bootupd.git && \
    mkdir -p /boot/efi && \
    touch /boot/efi/dummy.{dtb,dat,efi,elf,bin}

WORKDIR bootupd
RUN export CARGO_HOME=/tmp/.cargo && \
    mkdir -p $CARGO_HOME && \
    cargo build --release
RUN make install

After building the image with bib, I don't see the files in /boot/efi.

[cgwalters]

Thanks for working on this!

[cgwalters]

A core design principle that bootupd tries to maintain is that the operating system payload is underneath /usr as much as possible.

Today in bootc (and prior ostree) we encourage /boot to be empty in the container image. bootupd takes care at build time of moving the payloads to /usr.

It also has infrastructure to keep track of the versions of things it installs so it knows when to update.

So the problem domain is just more complex than this.

[cgwalters]

There's more discussion in #766 (let's make that the canonical issue).

[say-paul]

Ran: #936 on my current implementation with an additional step of creating the files in /boot:

# Build from the current git into a c9s-bootc container image.
# Use e.g. --build-arg=base=quay.io/fedora/fedora-bootc:41 to target
# Fedora or another base image instead.
#
ARG base=quay.io/centos-bootc/centos-bootc:stream9

FROM $base as build
# This installs our package dependencies, and we want to cache it independently of the rest.
# Basically we don't want changing a .rs file to blow out the cache of packages.
RUN <<EORUN
set -xeuo pipefail
dnf -y install cargo git openssl-devel
EORUN

RUN mkdir -p /boot/firmware/efi && \
    touch /boot/firmware/efi/dummy.{dtb,dat,efi,elf,bin}
# Now copy the source
COPY . /build
WORKDIR /build
# See https://www.reddit.com/r/rust/comments/126xeyx/exploring_the_problem_of_faster_cargo_docker/
# We aren't using the full recommendations there, just the simple bits.
RUN --mount=type=cache,target=/build/target --mount=type=cache,target=/var/roothome \
    make && make install-all DESTDIR=/out
  
# FROM $base
# # Clean out the default to ensure we're using our updated content
# RUN rpm -e bootupd
# COPY --from=build /out/ /
# # Sanity check this too
# RUN bootc container lint --fatal-warnings
  
    

From #755

Content shipped in an RPM with files in /boot that are present in the initial base image build where bootupctl backend generate-update-payload is invoked

The build container has the dummy files:

bash-5.2# ls /boot/firmware/efi/          
dummy.bin  dummy.dat  dummy.dtb  dummy.efi  dummy.elf

but the final image that is built using the COPY --from=build /out/ / doesn't have the files, 😕

[say-paul]

I think this function can be modified into a generic one to copy files, but I am not sure how to trigger it from the container build process

[cgwalters]

but the final image that is built using the COPY --from=build /out/ / doesn't have the files, 😕

The build image is only intended to hold the result of building the bootupd binary itself. If you want to modify the final image for other reasons (e.g. injecting kernel arguments, adding debug tools or other configuration, and as in this case inject arbitrary other content) then you should put it on the last stage.

Or better even - don't modify the dockerfile, make a new one and have it be FROM localhost/bootupd.

[cgwalters]

My Containerfile looks like this:

Can you try out #936 ? It has several advantages, but the biggest will be that you can just podman build it and it will do incremental builds from what's in local git, instead of hardcoding your repo.

[cgwalters]

Thank you for working on this. I propose that we continue design discussion (e.g. what the interface should look like) in #766 (comment) and I'll comment on the PR for code review.

But a specific downside of what you're doing here is that it would require all tools calling bootupd to be changed (particularly bootc) and to have specific knowledge of what should be installed.

Whereas the goal of bootupd is to abstract that from callers (like bootc) - the person building the container extends the payload, and bootc shouldn't need to know it's copying more files to the ESP.

Also, crucially, my proposal there would also transparently handle updates, which is an important goal.

[say-paul]

It should now stage the updates for any user input path,
I think we have to additionally chain this in efi::{install,run_update}
Its very similar to generate_update_metadata but with an user_input.

Addressed

[travier]

A lot of questions are unanswered here:

• Is those firmware files versioned?
• Is it always installed under the same name?
• Is it tied to a kernel version?
• What happens when we rollback?

Overall, I would suggest we work toward #926 which should get us to #766.

[say-paul]

The goal has changed with iterations of the idea and other effort on: #938 , So I updated the PR's main goal on my first comment

Is those firmware files versioned?

Yes, i can keep checks (rpm -qa) if we want to harden it.

Is it always installed under the same name?

I dont understand, User can extend _any_payload
example:
user can do this:

RUN dnf install -y uboot-images-armv8  # which installs  /usr/lib/uboot/rpi4/Uboot.bin
RUN bootupctl backend extend-payload-to-esp  /usr/lib/uboot/rpi4

Is it tied to a kernel version?

I don't know, since the command is flexible to extend anything , do we need some sort of check?

What happens when we rollback?

Since this integrates with bootupd, by moving the files to /usr/lib/bootupd/updates/ , I think(<100% sure) rollback will work....!!!

Overall, I would suggest we work toward #926 which should get us to #766.

Here's how it plays in with #926 : bootupctl backend extend-payload-to-esp /usr/lib/uboot/rpi4 can put the files in the necessary format as laid out by #926 so that generate-update-metadata can read from it.(some modifications required in the PR)

[say-paul]

This should now be inline with to what is required by #926

[say-paul]

We need to teach generate-update-metadata in #938 to read from /firmware/
and stage it for updates:
bootup extend-payload-to-esp /boot/efi to extend firmware to /usr usecase - > files installed in /bbot/efi by bcm2711-firmware.rpm
and also:
bootup extend-payload-to-esp /usr/share/uboot/rpi_arm64 -> use-case: copy /usr/share/uboot/rpi_arm64/rpi-u-boot.bin installed by uboot-images-armv8.rpm

[say-paul]

@travier please let me know if I am missing anything.