Docker build #486
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| **/.git | ||
| bazel-* |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| # Configuration related to self-hosted runner. | ||
| self-hosted-runner: | ||
| # Labels of self-hosted runner in array of strings. | ||
| labels: | ||
| - ubuntu-22.04-32core-graph-team-amd64 | ||
| - ubuntu-22.04-32core-graph-team-arm64 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,156 @@ | ||
| name: "Release Docker images" | ||
|
|
||
| on: | ||
| push: | ||
| branches: ["main"] | ||
| tags: | ||
| - "v*" | ||
| env: | ||
| REGISTRY_IMAGE: sourcegraph/scip-clang | ||
|
|
||
| jobs: | ||
| build-binaries: | ||
| strategy: | ||
| fail-fast: false | ||
|
There was a problem hiding this comment. Why is this set to |
||
| matrix: | ||
| include: | ||
| - os: ubuntu-22.04-32core-graph-team-amd64 | ||
| platform: linux/amd64 | ||
| binary-name: scip-clang-x86_64-linux | ||
| - os: ubuntu-22.04-32core-graph-team-arm64 | ||
| platform: linux/arm64 | ||
| binary-name: scip-clang-arm64-linux | ||
| runs-on: ${{ matrix.os }} | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - uses: bazel-contrib/[email protected] | ||
| with: | ||
| bazelisk-cache: true | ||
| disk-cache: ${{ github.workflow }}-${{ matrix.binary-name }} | ||
| repository-cache: true | ||
|
|
||
| - name: Build binary | ||
| run: bazel build //indexer:scip-clang --config release | ||
|
|
||
| - name: Upload artifacts | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| path: bazel-bin/indexer/scip-clang | ||
| name: ${{ matrix.binary-name }} | ||
| if-no-files-found: error | ||
|
There was a problem hiding this comment. Baffling that we have to specify this when literally specifying a There was a problem hiding this comment. the upload/download artifact github actions are the bane of my existence – they have a bunch of logic that is not super obvious, and given the iteration loop on GHA, make you furious |
||
|
|
||
| build-docker: | ||
| needs: [build-binaries] | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| include: | ||
| - os: ubuntu-22.04 | ||
| platform: linux/amd64 | ||
| binary-name: scip-clang-x86_64-linux | ||
| - os: ubuntu-22.04-arm | ||
| platform: linux/arm64 | ||
| binary-name: scip-clang-arm64-linux | ||
| runs-on: ${{ matrix.os }} | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Prepare | ||
| run: | | ||
| platform=${{ matrix.platform }} | ||
| echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV | ||
|
|
||
| - name: Docker meta | ||
| id: meta | ||
| uses: docker/metadata-action@v5 | ||
| with: | ||
| images: ${{ env.REGISTRY_IMAGE }} | ||
|
Comment on lines
+64
to
+68
There was a problem hiding this comment. Could you clarify why this step is before the Login step below? There was a problem hiding this comment. They can go in either order – this order is straight from their docs. |
||
|
|
||
| - name: Login to Docker Hub | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| username: ${{ secrets.DOCKER_USERNAME }} | ||
| password: ${{ secrets.DOCKER_PASSWORD }} | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Download pre-built binary | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| name: ${{ matrix.binary-name }} | ||
| path: /tmp/binary | ||
|
|
||
| - run: cp /tmp/binary/scip-clang ./scip-clang && ls -l ./scip-clang | ||
|
Comment on lines
+83
to
+85
There was a problem hiding this comment. I'm a bit confused. The artifact name ends up being There was a problem hiding this comment. Because we are downloading one artifact, the download-artifact action will just inline the contents of the artifact (which contain a single |
||
|
|
||
| - name: Build and push by digest | ||
| id: build-images | ||
| uses: docker/build-push-action@v6 | ||
| with: | ||
| context: . | ||
|
There was a problem hiding this comment. Would be good to add a comment on why this is necessary. I read the official docs, but couldn't quite follow as to why this argument is necessary. https://github.com/docker/build-push-action?tab=readme-ov-file#git-context There was a problem hiding this comment.
As we inject a non-tracked file, we need to explicitly use folder level context (instead of context which is just I'll add a comment |
||
| platforms: ${{ matrix.platform }} | ||
| labels: ${{ steps.meta.outputs.labels }} | ||
| tags: ${{ env.REGISTRY_IMAGE }} | ||
| outputs: type=image,push-by-digest=true,name-canonical=true,push=true | ||
|
|
||
| - name: Export digest | ||
| run: | | ||
| mkdir -p ${{ runner.temp }}/digests | ||
| digest="${{ steps.build-images.outputs.digest }}" | ||
| touch "${{ runner.temp }}/digests/${digest#sha256:}" | ||
|
|
||
| - name: Upload digest | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: digests-${{ env.PLATFORM_PAIR }} | ||
| path: ${{ runner.temp }}/digests/* | ||
| if-no-files-found: error | ||
| retention-days: 1 | ||
|
|
||
| merge-docker: | ||
| runs-on: ubuntu-latest | ||
| needs: [build-docker] | ||
| steps: | ||
| - name: Download digests | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| path: ${{ runner.temp }}/digests | ||
| pattern: digests-* | ||
| merge-multiple: true | ||
|
|
||
| - name: Login to Docker Hub | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| username: ${{ secrets.DOCKER_USERNAME }} | ||
| password: ${{ secrets.DOCKER_PASSWORD }} | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Snapshot metadata | ||
| if: github.ref == 'refs/heads/main' | ||
| uses: docker/metadata-action@v5 | ||
| with: | ||
| images: | | ||
| ${{ env.REGISTRY_IMAGE }} | ||
| tags: | | ||
| type=raw,value=latest-snapshot | ||
|
|
||
| - name: Tag metadata | ||
| if: startsWith(github.ref, 'refs/tags/v') | ||
| uses: docker/metadata-action@v5 | ||
| with: | ||
| images: | | ||
| ${{ env.REGISTRY_IMAGE }} | ||
| tags: | | ||
| type=raw,value=latest | ||
| type=raw,value=${{ env.PATCH }} | ||
| type=raw,value=${{ env.MINOR }} | ||
| type=raw,value=${{ env.MAJOR }} | ||
|
Comment on lines
+148
to
+150
There was a problem hiding this comment. I don't see these environment variables being set in this workflow file. Are they being set by some other workflow and being injected? |
||
|
|
||
| - name: Create manifest list and push | ||
| working-directory: ${{ runner.temp }}/digests | ||
| run: | | ||
| docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \ | ||
|
There was a problem hiding this comment. Would be good to add a comment describing what sets this environment variable; it's not obvious by looking at the configuration. Could we perhaps split this logic over multiple lines add a comment or two describing the format/conversion with 1 example? There was a problem hiding this comment. So this, and several other things that might be confusing come directly from Docker's github actions (most copied verbatim): https://docs.docker.com/build/ci/github-actions/multi-platform/#distribute-build-across-multiple-runners I will add a comment about the source of those steps, but I consider them internal details of the steps provided by the various docker actions – as such I would rather not deviate textually from the actions they have in their docs – all those env variables are set and processed by their custom actions. |
||
| $(printf '${{ env.REGISTRY_IMAGE }}@sha256:%s ' *) | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| FROM ubuntu:22.04 as indexer | ||
|
|
||
| RUN apt-get update && apt-get install -y curl libc6-dev python3 build-essential ninja-build git cmake | ||
|
|
||
| COPY ./scip-clang /usr/bin/scip-clang | ||
| RUN chmod +x /usr/bin/scip-clang && chown $(whoami) /usr/bin/scip-clang | ||
|
|
||
| WORKDIR /sources | ||
|
|
||
| ENTRYPOINT [ "scip-clang" ] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -210,22 +210,22 @@ def fetch_direct_dependencies(): | |
| http_archive( | ||
| name = "actionlint_darwin_arm64", | ||
| build_file = "@scip_clang//third_party:actionlint.BUILD", | ||
| sha256 = "2693315b9093aeacb4ebd91a993fea54fc215057bf0da2659056b4bc033873db", | ||
| urls = ["https://github.com/rhysd/actionlint/releases/download/v1.7.7/actionlint_1.7.7_darwin_arm64.tar.gz"], | ||
|
There was a problem hiding this comment. Could you briefly describe why we needed to bump actionlint here? Did you hit a bug? There was a problem hiding this comment. https://github.com/rhysd/actionlint/releases/tag/v1.7.7 Main reason that they didn't support |
||
| ) | ||
|
|
||
| http_archive( | ||
| name = "actionlint_linux_amd64", | ||
| build_file = "@scip_clang//third_party:actionlint.BUILD", | ||
| sha256 = "023070a287cd8cccd71515fedc843f1985bf96c436b7effaecce67290e7e0757", | ||
| urls = ["https://github.com/rhysd/actionlint/releases/download/v1.7.7/actionlint_1.7.7_linux_amd64.tar.gz"], | ||
| ) | ||
|
|
||
| http_archive( | ||
| name = "actionlint_linux_arm64", | ||
| build_file = "@scip_clang//third_party:actionlint.BUILD", | ||
| sha256 = "401942f9c24ed71e4fe71b76c7d638f66d8633575c4016efd2977ce7c28317d0", | ||
| urls = ["https://github.com/rhysd/actionlint/releases/download/v1.7.7/actionlint_1.7.7_linux_arm64.tar.gz"], | ||
| ) | ||
|
|
||
| http_archive( | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.