LLVM fails some PassBuilderCallbacks tests with 12.0.0-rc1

[marv]

Bugzilla Link	48992
Resolution	FIXED
Resolved on	Jun 04, 2021 11:42
Version	unspecified
OS	Linux
Blocks	#48661
CC	@foutrelis,@LW-archlinux,@xhochy,@serge-sans-paille,@tstellar,@yuanfang-chen
Fixed by commit(s)	3d3abc2 328a6ec

Extended Description

On Exherbo I see the following test failures with 12.0.0-rc1:

Failed Tests (10):
LLVM-Unit :: IR/./IRTests/CGSCCCallbacksTest.InstrumentedInvalidatingPasses
LLVM-Unit :: IR/./IRTests/CGSCCCallbacksTest.InstrumentedPasses
LLVM-Unit :: IR/./IRTests/CGSCCCallbacksTest.InstrumentedSkippedPasses
LLVM-Unit :: IR/./IRTests/FunctionCallbacksTest.InstrumentedPasses
LLVM-Unit :: IR/./IRTests/FunctionCallbacksTest.InstrumentedSkippedPasses
LLVM-Unit :: IR/./IRTests/LoopCallbacksTest.InstrumentedInvalidatingLoopNestPasses
LLVM-Unit :: IR/./IRTests/LoopCallbacksTest.InstrumentedInvalidatingPasses
LLVM-Unit :: IR/./IRTests/LoopCallbacksTest.InstrumentedPasses
LLVM-Unit :: IR/./IRTests/LoopCallbacksTest.InstrumentedSkippedPasses
LLVM-Unit :: IR/./IRTests/ModuleCallbacksTest.InstrumentedSkippedPasses

Testing Time: 205.97s
Unsupported : 1522
Passed : 40384
Expectedly Failed: 143
Failed : 10

The tests failures all look like this:

[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from LoopCallbacksTest
[ RUN ] LoopCallbacksTest.InstrumentedInvalidatingPasses
unknown file: Failure

Unexpected mock function call - returning directly.
Function call: runBeforeNonSkippedPass("PassManagerllvm::Function", 8-byte object <10-98 B3-CF 9C-55 00-00>)
Google Mock tried the following 4 expectations, but none matched:

/var/tmp/paludis/build/dev-lang-llvm-12.0.0_rc1/work/llvm-project/llvm/unittests/IR/PassBuilderCallbacksTest.cpp:390: tried expectation #​0: EXPECT_CALL(*this, runBeforeNonSkippedPass(Not(HasNameRegex("Mock")), HasName(IRName)))...
Expected arg #​1: has name ""
Actual: 8-byte object <30-98 B3-CF 9C-55 00-00>, has name ''
Expected: to be called any number of times
Actual: called once - satisfied and active
/var/tmp/paludis/build/dev-lang-llvm-12.0.0_rc1/work/llvm-project/llvm/unittests/IR/PassBuilderCallbacksTest.cpp:390: tried expectation #​1: EXPECT_CALL(*this, runBeforeNonSkippedPass(Not(HasNameRegex("Mock")), HasName(IRName)))...
Expected arg #​1: has name "foo"
Actual: 8-byte object <30-98 B3-CF 9C-55 00-00>, has name ''
Expected: to be called any number of times
Actual: never called - satisfied and active
/var/tmp/paludis/build/dev-lang-llvm-12.0.0_rc1/work/llvm-project/llvm/unittests/IR/PassBuilderCallbacksTest.cpp:390: tried expectation #​2: EXPECT_CALL(*this, runBeforeNonSkippedPass(Not(HasNameRegex("Mock")), HasName(IRName)))...
Expected arg #​1: has name "loop"
Actual: 8-byte object <30-98 B3-CF 9C-55 00-00>, has name ''
Expected: to be called any number of times
Actual: never called - satisfied and active
/var/tmp/paludis/build/dev-lang-llvm-12.0.0_rc1/work/llvm-project/llvm/unittests/IR/PassBuilderCallbacksTest.cpp:902: tried expectation #​3: EXPECT_CALL(CallbacksHandle, runBeforeNonSkippedPass(HasNameRegex("MockPassHandle"), HasName("loop")))...
Expected arg #​0: has name regex "MockPassHandle"
Actual: "PassManagerllvm::Function", has name 'PassManagerllvm::Function'
Expected arg #​1: has name "loop"
Actual: 8-byte object <30-98 B3-CF 9C-55 00-00>, has name ''
Expected: to be called once
Actual: never called - unsatisfied and active
unknown file: Failure

Unexpected mock function call - returning directly.
Function call: runBeforeNonSkippedPass("FunctionToLoopPassAdaptor", 8-byte object <30-98 B3-CF 9C-55 00-00>)
Google Mock tried the following 4 expectations, but none matched:

/var/tmp/paludis/build/dev-lang-llvm-12.0.0_rc1/work/llvm-project/llvm/unittests/IR/PassBuilderCallbacksTest.cpp:390: tried expectation #​0: EXPECT_CALL(*this, runBeforeNonSkippedPass(Not(HasNameRegex("Mock")), HasName(IRName)))...
Expected arg #​1: has name ""
Actual: 8-byte object <80-B2 B3-CF 9C-55 00-00>, has name ''
Expected: to be called any number of times
Actual: called once - satisfied and active
/var/tmp/paludis/build/dev-lang-llvm-12.0.0_rc1/work/llvm-project/llvm/unittests/IR/PassBuilderCallbacksTest.cpp:390: tried expectation #​1: EXPECT_CALL(*this, runBeforeNonSkippedPass(Not(HasNameRegex("Mock")), HasName(IRName)))...
Expected arg #​1: has name "foo"
Actual: 8-byte object <80-B2 B3-CF 9C-55 00-00>, has name ''
Expected: to be called any number of times
Actual: never called - satisfied and active
/var/tmp/paludis/build/dev-lang-llvm-12.0.0_rc1/work/llvm-project/llvm/unittests/IR/PassBuilderCallbacksTest.cpp:390: tried expectation #​2: EXPECT_CALL(*this, runBeforeNonSkippedPass(Not(HasNameRegex("Mock")), HasName(IRName)))...
Expected arg #​1: has name "loop"
Actual: 8-byte object <80-B2 B3-CF 9C-55 00-00>, has name ''
Expected: to be called any number of times
Actual: never called - satisfied and active
/var/tmp/paludis/build/dev-lang-llvm-12.0.0_rc1/work/llvm-project/llvm/unittests/IR/PassBuilderCallbacksTest.cpp:902: tried expectation #​3: EXPECT_CALL(CallbacksHandle, runBeforeNonSkippedPass(HasNameRegex("MockPassHandle"), HasName("loop")))...
Expected arg #​0: has name regex "MockPassHandle"
Actual: "FunctionToLoopPassAdaptor", has name 'FunctionToLoopPassAdaptor'
Expected arg #​1: has name "loop"
Actual: 8-byte object <80-B2 B3-CF 9C-55 00-00>, has name ''
Expected: to be called once
Actual: never called - unsatisfied and active
unknown file: Failure

[...]

The name is always "" instead of the expected name.

Full log can be found here: https://dev.exherbo.org/~marv/1611936999-install-dev-lang_llvm-12.0.0_rc1:12::arbor.out

Since all commits touching llvm/unittests/IR/PassBuilderCallbacksTest.cpp are prefixed with "NewPM" I tried building with -DENABLE_EXPERIMENTAL_NEW_PASS_MANAGER=ON as well, but with the same result.

Any help is much appreciated

[yuanfang-chen]

Any easy way to reproduce this locally? I could not reproduce this with gcc-10.1 + ubuntu18.04 + llvm12.0.0-rc1.

[LW-archlinux]

Also occurs when building llvm 13 (trunk) on archlinux, gcc 10.2 .

[xhochy]

On conda-forge, we are seeing the same issue. For example for the IRTests the following tests fail:

[ FAILED ] ModuleCallbacksTest.InstrumentedSkippedPasses
[ FAILED ] FunctionCallbacksTest.InstrumentedPasses
[ FAILED ] FunctionCallbacksTest.InstrumentedSkippedPasses
[ FAILED ] LoopCallbacksTest.InstrumentedPasses
[ FAILED ] LoopCallbacksTest.InstrumentedInvalidatingPasses
[ FAILED ] LoopCallbacksTest.InstrumentedInvalidatingLoopNestPasses
[ FAILED ] LoopCallbacksTest.InstrumentedSkippedPasses
[ FAILED ] CGSCCCallbacksTest.InstrumentedPasses
[ FAILED ] CGSCCCallbacksTest.InstrumentedInvalidatingPasses
[ FAILED ] CGSCCCallbacksTest.InstrumentedSkippedPasses

You can find the full build log at https://gist.github.com/xhochy/f85e61c0e0b1ddbc1613f1945822703d and the pull request to update LLVM is at conda-forge/llvmdev-feedstock#116

[marv]

I figured out that the test failures only happen when building the dynamic LLVM library, but not linking against it.
So with

-DLLVM_BUILD_LLVM_DYLIB:BOOL=TRUE
-DLLVM_LINK_LLVM_DYLIB:BOOL=TRUE

the tests fail as described above, while they pass when building with

-DLLVM_LINK_LLVM_DYLIB:BOOL=FALSE

But the overall package size increases quite a bit, which is why we'd like to stick with -DLLVM_LINK_LLVM_DYLIB:BOOL=TRUE

I could imagine that googlemock might not work as expected when linking dynamically against the shared LLVM library or something in that direction?

[tstellar]

This patch should fix the issue: https://reviews.llvm.org/D101972

[tstellar]

*** Bug llvm/llvm-bugzilla-archive#50237 has been marked as a duplicate of this bug. ***

[tstellar]

Merged: 328a6ec

[tstellar]

mentioned in issue #48661

[tstellar]

mentioned in issue llvm/llvm-bugzilla-archive#50237