Description
Description of the bug:
We started seeing the following errors during dex merging when using D8
23:01:25 [Bazel] Error in bazel-out/darwin_x86_64-fastbuild/bin/apps/hermosa/matador-services/settings/dexsplits/hermosa-settings-service-debug/9.shard.zip:kotlinx/coroutines/rx3/RxSingleKt$$ExternalSyntheticLambda0.class.dex:
23:01:25 [Bazel] Attempt at compiling intermediate artifact without its context
23:01:25 [Bazel] Merge failed: Compilation failed to complete, origin: bazel-out/darwin_x86_64-fastbuild/bin/apps/hermosa/matador-services/settings/dexsplits/hermosa-settings-service-debug/9.shard.zip:kotlinx/coroutines/rx3/RxSingleKt$$ExternalSyntheticLambda0.class.dex
23:01:25 [Bazel] Exception in thread "main" java.lang.RuntimeException: Merge failed: Compilation failed to complete, origin: bazel-out/darwin_x86_64-fastbuild/bin/apps/hermosa/matador-services/settings/dexsplits/hermosa-settings-service-debug/9.shard.zip:kotlinx/coroutines/rx3/RxSingleKt$$ExternalSyntheticLambda0.class.dex
23:01:25 [Bazel] at com.google.devtools.build.android.r8.DexFileMerger.main(DexFileMerger.java:412)
23:01:25 [Bazel] Caused by: com.android.tools.r8.CompilationFailedException: Compilation failed to complete, origin: bazel-out/darwin_x86_64-fastbuild/bin/apps/hermosa/matador-services/settings/dexsplits/hermosa-settings-service-debug/9.shard.zip:kotlinx/coroutines/rx3/RxSingleKt$$ExternalSyntheticLambda0.class.dex
23:01:25 [Bazel] at Version.fakeStackEntry(Version_8.1.56.java:0)
23:01:25 [Bazel] at com.android.tools.r8.M.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:5)
23:01:25 [Bazel] at com.android.tools.r8.utils.R0.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:81)
23:01:25 [Bazel] at com.android.tools.r8.utils.R0.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:32)
23:01:25 [Bazel] at com.android.tools.r8.utils.R0.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:31)
23:01:25 [Bazel] at com.android.tools.r8.utils.R0.b(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:2)
23:01:25 [Bazel] at com.android.tools.r8.D8.run(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:6)
23:01:25 [Bazel] at com.google.devtools.build.android.r8.DexFileMerger.run(DexFileMerger.java:395)
23:01:25 [Bazel] at com.google.devtools.build.android.r8.DexFileMerger.main(DexFileMerger.java:409)
23:01:25 [Bazel] Caused by: com.android.tools.r8.utils.b: Attempt at compiling intermediate artifact without its context
23:01:25 [Bazel] at com.android.tools.r8.utils.O2.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:21)
23:01:25 [Bazel] at com.android.tools.r8.utils.O2.error(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:1)
23:01:25 [Bazel] at com.android.tools.r8.synthesis.D.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:59)
23:01:25 [Bazel] at com.android.tools.r8.D8.d(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:26)
23:01:25 [Bazel] at com.android.tools.r8.D8.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:20)
23:01:25 [Bazel] at com.android.tools.r8.utils.R0.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:28)
23:01:25 [Bazel] ... 4 more
23:01:25 [Bazel] Suppressed: java.lang.RuntimeException: com.android.tools.r8.utils.b: Attempt at compiling intermediate artifact without its context
23:01:25 [Bazel] at com.android.tools.r8.utils.O2.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:29)
23:01:25 [Bazel] at com.android.tools.r8.dex.k.a(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:225)
23:01:25 [Bazel] at com.android.tools.r8.D8.d(R8_8.1.56_756d1f50f618dd1c39c000f11defb367a21e9e866e3401b884be16c0950f6f79:149)
23:01:25 [Bazel] ... 6 more
23:01:25 [Bazel] Caused by: [CIRCULAR REFERENCE: com.android.tools.r8.utils.b: Attempt at compiling intermediate artifact without its context]
I did some investigation and the META-INF/synthetic-contexts.map in the .jar.dex.zip is incomplete. It looks under some circumstances an older version of D8 is being used during dexing. The version of R8 being used, is an older version which does not output information about synthetic classes and their contexts. The version that should be used should be the one pointed by android_gmaven_r8, but instead is the one bundled by the @maven_android//:com_android_tools_build_builder_file dep which was added as dependency in this commit bazelbuild/bazel@aaf94c6#diff-bdb5a620f62ba1e7228456cde29d508737679c9196143db3641b22bb3ca2fd2aR128. If com_android_tools_build_builde happens to appear before than the R8 version coming from android_gmave_r8 dep in the dexer classpath, then dexmerging is likely to fail due to missing synthetic classes info.
The r8 target shipped with the android tools depends on the gradle builder jar via //src/tools/android/java/com/google/devtools/build/android:all_android_tools which in turn transitively depends on the gradle jar. The gradle dependency needs to be teased out in order to avoid causing classpath conflicts.
The version used seems to be non-deterministic and depends on which commit we are building our app . Furthermore we've only encountered this bug in our linux builds and we have not been able to repro on Mac builds.
Which category does this issue belong to?
Android
What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
This bug is not deterministic and we are only able to repro at specific commits when building our app, but its easy to check the bundled version of R8 in the gradle jar. Furthermore we've only encountered this bug in our linux builds and we have not been able to repro on Mac builds.
Which operating system are you running Bazel on?
Linux
What is the output of bazel info release?
release 7.0.2.13 (internal release)
If bazel info release returns development version or (@non-git), tell us how you built Bazel.
Built from internal forked version of bazel. Forked at this commit from upstream bazelbuild/bazel@252d363
What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?
No response
Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.
Have you found anything relevant by searching the web?
No response
Any other information, logs, or outputs that you want to share?
No response