[IR] Only allow lifetime.start/end on allocas #149310
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@efriedma-quic @fhahn @preames @dtcxzyw I'd like some early feedback on this before I sink time into updating more tests. Are there any concerns with limiting lifetime markers to allocas only? |
|
Generally, it makes sense to me. I also thought about this last year: dtcxzyw/llvm-tools@c5ce24a Unfortunately, the code reminds me that lifetime markers are also used on BTW, does the order of lifetime markers matter? I mean the start-end sequence should look like a balanced bracket sequence. See also https://github.com/dtcxzyw/llvm-project/pull/2/files#diff-afb2b04eff951cb67d214bc6dfa62a087a72591451817dba7929b48c5fe635f9. |
Hm, I've not encountered this on any clang tests or in llvm-test-suite. I wonder whether this is an older issue that has since been fixed?
I don't think it matters right now as long as the interference graph is preserved. We'd want this property though for a more intrusive redesign of lifetime intrinsics, along the lines of what is discussed in #45725. |
Looking a bit closer, it looks like even though this is in the |
nikic
force-pushed
the
lifetime-alloca-only-2
branch
from
0fa2a96 to
955b133
Compare
dtcxzyw
reviewed
Jul 18, 2025
There was a problem hiding this comment.
dead code:
llvm-project/llvm/lib/Transforms/Utils/MemoryTaggingSupport.cpp
Lines 237 to 239 in 7d040d4
Removed in 0121314. |
llvmbot
added
backend:AArch64
backend:X86
llvm:codegen
llvm:globalisel
mlir:llvm
mlir
llvm:instcombine
|
@llvm/pr-subscribers-llvm-ir @llvm/pr-subscribers-backend-aarch64 Author: Nikita Popov (nikic) Changeslifetime.start and lifetime.end are primarily intended for use on allocas, to enable stack coloring and other liveness optimizations. This is necessary because all (static) allocas are hoisted into the entry block, so lifetime markers are necessary to convey the actual lifetimes. However, lifetime.start and lifetime.end are currently allowed to be used on non-alloca pointers. We don't actually do this in practice, but just the mere fact that this is possible breaks the core purpose of the lifetime markers, which is stack coloring of allocas. Stack coloring can only work correctly if all lifetime markers for an alloca are analyzable.
I don't think there is any way to have coherent semantics for lifetime markers on allocas, while also permitting them on arbitrary pointer values. This PR restricts lifetimes to operate on allocas only. As a followup, I will also drop the size argument, which is superfluous if we always operate on an alloca. (This change also renders various code handling lifetime markers on non-alloca dead. I plan to clean up that kind of code after dropping the size argument as well.) In practice, I've only found a few places that currently produce lifetimes on non-allocas:
Patch is 157.93 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/149310.diff 63 Files Affected:
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index 371f356c80b0a..aacdf914ba61d 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -26634,19 +26634,14 @@ Arguments:
The first argument is a constant integer representing the size of the
object, or -1 if it is variable sized. The second argument is a pointer
-to the object.
+to an ``alloca`` instruction.
Semantics:
""""""""""
-If ``ptr`` is a stack-allocated object and it points to the first byte of
-the object, the object is initially marked as dead.
-``ptr`` is conservatively considered as a non-stack-allocated object if
-the stack coloring algorithm that is used in the optimization pipeline cannot
-conclude that ``ptr`` is a stack-allocated object.
-
-After '``llvm.lifetime.start``', the stack object that ``ptr`` points is marked
-as alive and has an uninitialized value.
+The stack-allocated object that ``ptr`` points to is initially marked as dead.
+After '``llvm.lifetime.start``', the stack object is marked as alive and has an
+uninitialized value.
The stack object is marked as dead when either
:ref:`llvm.lifetime.end <int_lifeend>` to the alloca is executed or the
function returns.
@@ -26656,11 +26651,6 @@ After :ref:`llvm.lifetime.end <int_lifeend>` is called,
The second '``llvm.lifetime.start``' call marks the object as alive, but it
does not change the address of the object.
-If ``ptr`` is a non-stack-allocated object, it does not point to the first
-byte of the object or it is a stack object that is already alive, it simply
-fills all bytes of the object with ``poison``.
-
-
.. _int_lifeend:
'``llvm.lifetime.end``' Intrinsic
@@ -26684,24 +26674,16 @@ Arguments:
The first argument is a constant integer representing the size of the
object, or -1 if it is variable sized. The second argument is a pointer
-to the object.
+to an ``alloca`` instruction.
Semantics:
""""""""""
-If ``ptr`` is a stack-allocated object and it points to the first byte of the
-object, the object is dead.
-``ptr`` is conservatively considered as a non-stack-allocated object if
-the stack coloring algorithm that is used in the optimization pipeline cannot
-conclude that ``ptr`` is a stack-allocated object.
+The stack-allocated object that ``ptr`` points to becomes dead after the call
+to this intrinsic.
Calling ``llvm.lifetime.end`` on an already dead alloca is no-op.
-If ``ptr`` is a non-stack-allocated object or it does not point to the first
-byte of the object, it is equivalent to simply filling all bytes of the object
-with ``poison``.
-
-
'``llvm.invariant.start``' Intrinsic
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
index 66ecc69c9874d..82c94b689aca6 100644
--- a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
+++ b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
@@ -7116,9 +7116,11 @@ Error BitcodeReader::materializeModule() {
if (CallInst *CI = dyn_cast<CallInst>(U))
UpgradeIntrinsicCall(CI, I.second);
}
- if (!I.first->use_empty())
- I.first->replaceAllUsesWith(I.second);
- I.first->eraseFromParent();
+ if (I.first != I.second) {
+ if (!I.first->use_empty())
+ I.first->replaceAllUsesWith(I.second);
+ I.first->eraseFromParent();
+ }
}
UpgradedIntrinsics.clear();
diff --git a/llvm/lib/CodeGen/SafeStack.cpp b/llvm/lib/CodeGen/SafeStack.cpp
index 996207034d076..908ed96172615 100644
--- a/llvm/lib/CodeGen/SafeStack.cpp
+++ b/llvm/lib/CodeGen/SafeStack.cpp
@@ -614,6 +614,13 @@ Value *SafeStack::moveStaticAllocasToUnsafeStack(
Use &U = *AI->use_begin();
Instruction *User = cast<Instruction>(U.getUser());
+ // Drop lifetime markers now that this is no longer an alloca.
+ // SafeStack has already performed its own stack coloring.
+ if (User->isLifetimeStartOrEnd()) {
+ User->eraseFromParent();
+ continue;
+ }
+
Instruction *InsertBefore;
if (auto *PHI = dyn_cast<PHINode>(User))
InsertBefore = PHI->getIncomingBlock(U)->getTerminator();
diff --git a/llvm/lib/IR/AutoUpgrade.cpp b/llvm/lib/IR/AutoUpgrade.cpp
index 86285a03c66bb..28ed1e520ce52 100644
--- a/llvm/lib/IR/AutoUpgrade.cpp
+++ b/llvm/lib/IR/AutoUpgrade.cpp
@@ -1310,6 +1310,18 @@ static bool upgradeIntrinsicFunction1(Function *F, Function *&NewFn,
return true;
}
break;
+ case 'l':
+ if (Name.starts_with("lifetime.start") ||
+ Name.starts_with("lifetime.end")) {
+ // Unless remangling is required, do not upgrade the function declaration,
+ // but do upgrade the calls.
+ if (auto Result = llvm::Intrinsic::remangleIntrinsicFunction(F))
+ NewFn = *Result;
+ else
+ NewFn = F;
+ return true;
+ }
+ break;
case 'm': {
// Updating the memory intrinsics (memcpy/memmove/memset) that have an
// alignment parameter to embedding the alignment as an attribute of
@@ -1629,7 +1641,6 @@ bool llvm::UpgradeIntrinsicFunction(Function *F, Function *&NewFn,
NewFn = nullptr;
bool Upgraded =
upgradeIntrinsicFunction1(F, NewFn, CanUpgradeDebugIntrinsicsToRecords);
- assert(F != NewFn && "Intrinsic function upgraded to the same function");
// Upgrade intrinsic attributes. This does not change the function.
if (NewFn)
@@ -4570,6 +4581,9 @@ void llvm::UpgradeIntrinsicCall(CallBase *CI, Function *NewFn) {
}
const auto &DefaultCase = [&]() -> void {
+ if (F == NewFn)
+ return;
+
if (CI->getFunctionType() == NewFn->getFunctionType()) {
// Handle generic mangling change.
assert(
@@ -5109,6 +5123,31 @@ void llvm::UpgradeIntrinsicCall(CallBase *CI, Function *NewFn) {
MTI->setSourceAlignment(Align->getMaybeAlignValue());
break;
}
+
+ case Intrinsic::lifetime_start:
+ case Intrinsic::lifetime_end: {
+ Value *Size = CI->getArgOperand(0);
+ Value *Ptr = CI->getArgOperand(1);
+ if (isa<AllocaInst>(Ptr)) {
+ DefaultCase();
+ return;
+ }
+
+ // Try to strip pointer casts, such that the lifetime works on an alloca.
+ Ptr = Ptr->stripPointerCasts();
+ if (isa<AllocaInst>(Ptr)) {
+ // Don't use NewFn, as we might have looked through an addrspacecast.
+ if (NewFn->getIntrinsicID() == Intrinsic::lifetime_start)
+ NewCall = Builder.CreateLifetimeStart(Ptr, cast<ConstantInt>(Size));
+ else
+ NewCall = Builder.CreateLifetimeEnd(Ptr, cast<ConstantInt>(Size));
+ break;
+ }
+
+ // Otherwise remove the lifetime marker.
+ CI->eraseFromParent();
+ return;
+ }
}
assert(NewCall && "Should have either set this variable or returned through "
"the default case");
@@ -5131,7 +5170,8 @@ void llvm::UpgradeCallsToIntrinsic(Function *F) {
UpgradeIntrinsicCall(CB, NewFn);
// Remove old function, no longer used, from the module.
- F->eraseFromParent();
+ if (F != NewFn)
+ F->eraseFromParent();
}
}
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index 8c8ed3c5e47ba..196a52cc07120 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -6679,6 +6679,11 @@ void Verifier::visitIntrinsicCall(Intrinsic::ID ID, CallBase &Call) {
"llvm.threadlocal.address operand isThreadLocal() must be true");
break;
}
+ case Intrinsic::lifetime_start:
+ case Intrinsic::lifetime_end:
+ Check(isa<AllocaInst>(Call.getArgOperand(1)),
+ "llvm.lifetime.start/end can only be used on alloca", &Call);
+ break;
};
// Verify that there aren't any unmediated control transfers between funclets.
diff --git a/llvm/lib/Target/SPIRV/SPIRVPrepareFunctions.cpp b/llvm/lib/Target/SPIRV/SPIRVPrepareFunctions.cpp
index 2bffbf73b574a..6766bd866cd2e 100644
--- a/llvm/lib/Target/SPIRV/SPIRVPrepareFunctions.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVPrepareFunctions.cpp
@@ -380,7 +380,7 @@ bool SPIRVPrepareFunctions::substituteIntrinsicCalls(Function *F) {
bool Changed = false;
const SPIRVSubtarget &STI = TM.getSubtarget<SPIRVSubtarget>(*F);
for (BasicBlock &BB : *F) {
- for (Instruction &I : BB) {
+ for (Instruction &I : make_early_inc_range(BB)) {
auto Call = dyn_cast<CallInst>(&I);
if (!Call)
continue;
@@ -408,12 +408,16 @@ bool SPIRVPrepareFunctions::substituteIntrinsicCalls(Function *F) {
if (!STI.isShader()) {
Changed |= toSpvOverloadedIntrinsic(
II, Intrinsic::SPVIntrinsics::spv_lifetime_start, {1});
+ } else {
+ II->eraseFromParent();
}
break;
case Intrinsic::lifetime_end:
if (!STI.isShader()) {
Changed |= toSpvOverloadedIntrinsic(
II, Intrinsic::SPVIntrinsics::spv_lifetime_end, {1});
+ } else {
+ II->eraseFromParent();
}
break;
case Intrinsic::ptr_annotation:
diff --git a/llvm/lib/Transforms/Coroutines/CoroEarly.cpp b/llvm/lib/Transforms/Coroutines/CoroEarly.cpp
index e279fec18bdbc..6561b1cd4ade1 100644
--- a/llvm/lib/Transforms/Coroutines/CoroEarly.cpp
+++ b/llvm/lib/Transforms/Coroutines/CoroEarly.cpp
@@ -170,6 +170,12 @@ void Lowerer::hidePromiseAlloca(CoroIdInst *CoroId, CoroBeginInst *CoroBegin) {
auto *PI = Builder.CreateIntrinsic(
Builder.getPtrTy(), Intrinsic::coro_promise, Arg, {}, "promise.addr");
PI->setCannotDuplicate();
+ // Remove lifetime markers, as these are only allowed on allocas.
+ for (User *U : make_early_inc_range(PA->users())) {
+ auto *I = cast<Instruction>(U);
+ if (I->isLifetimeStartOrEnd())
+ I->eraseFromParent();
+ }
PA->replaceUsesWithIf(PI, [CoroId](Use &U) {
bool IsBitcast = U == U.getUser()->stripPointerCasts();
bool IsCoroId = U.getUser() == CoroId;
diff --git a/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp b/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
index 5957940add577..fbaa651641566 100644
--- a/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
+++ b/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
@@ -3637,6 +3637,7 @@ void FunctionStackPoisoner::processStaticAllocas() {
"Variable descriptions relative to ASan stack base will be dropped");
// Replace Alloca instructions with base+offset.
+ SmallVector<Value *> NewAllocaPtrs;
for (const auto &Desc : SVD) {
AllocaInst *AI = Desc.AI;
replaceDbgDeclare(AI, LocalStackBaseAllocaPtr, DIB, DIExprFlags,
@@ -3645,6 +3646,7 @@ void FunctionStackPoisoner::processStaticAllocas() {
IRB.CreateAdd(LocalStackBase, ConstantInt::get(IntptrTy, Desc.Offset)),
AI->getType());
AI->replaceAllUsesWith(NewAllocaPtr);
+ NewAllocaPtrs.push_back(NewAllocaPtr);
}
// The left-most redzone has enough space for at least 4 pointers.
@@ -3694,6 +3696,15 @@ void FunctionStackPoisoner::processStaticAllocas() {
}
}
+ // Remove lifetime markers now that these are no longer allocas.
+ for (Value *NewAllocaPtr : NewAllocaPtrs) {
+ for (User *U : make_early_inc_range(NewAllocaPtr->users())) {
+ auto *I = cast<Instruction>(U);
+ if (I->isLifetimeStartOrEnd())
+ I->eraseFromParent();
+ }
+ }
+
SmallVector<uint8_t, 64> ShadowClean(ShadowAfterScope.size(), 0);
SmallVector<uint8_t, 64> ShadowAfterReturn;
@@ -3829,6 +3840,13 @@ void FunctionStackPoisoner::handleDynamicAllocaCall(AllocaInst *AI) {
Value *NewAddressPtr = IRB.CreateIntToPtr(NewAddress, AI->getType());
+ // Remove lifetime markers now that this is no longer an alloca.
+ for (User *U : make_early_inc_range(AI->users())) {
+ auto *I = cast<Instruction>(U);
+ if (I->isLifetimeStartOrEnd())
+ I->eraseFromParent();
+ }
+
// Replace all uses of AddessReturnedByAlloca with NewAddressPtr.
AI->replaceAllUsesWith(NewAddressPtr);
diff --git a/llvm/lib/Transforms/Scalar/InferAddressSpaces.cpp b/llvm/lib/Transforms/Scalar/InferAddressSpaces.cpp
index 66836ef05d5db..85ee824b67121 100644
--- a/llvm/lib/Transforms/Scalar/InferAddressSpaces.cpp
+++ b/llvm/lib/Transforms/Scalar/InferAddressSpaces.cpp
@@ -430,6 +430,8 @@ bool InferAddressSpacesImpl::rewriteIntrinsicOperands(IntrinsicInst *II,
}
case Intrinsic::lifetime_start:
case Intrinsic::lifetime_end: {
+ // Always force lifetime markers to work directly on the alloca.
+ NewV = NewV->stripPointerCasts();
Function *NewDecl = Intrinsic::getOrInsertDeclaration(
M, II->getIntrinsicID(), {NewV->getType()});
II->setArgOperand(1, NewV);
diff --git a/llvm/test/Analysis/BasicAA/modref.ll b/llvm/test/Analysis/BasicAA/modref.ll
index 0619f8e615b80..1aab28f3f1871 100644
--- a/llvm/test/Analysis/BasicAA/modref.ll
+++ b/llvm/test/Analysis/BasicAA/modref.ll
@@ -67,27 +67,33 @@ define i8 @test2a(ptr %P) {
ret i8 %A
}
-define void @test3(ptr %P, i8 %X) {
+define void @test3(i8 %X) {
; CHECK-LABEL: @test3(
-; CHECK-NEXT: [[P2:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 2
+; CHECK-NEXT: [[P:%.*]] = alloca i64, align 8
+; CHECK-NEXT: [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 2
; CHECK-NEXT: call void @llvm.lifetime.end.p0(i64 1, ptr [[P]])
; CHECK-NEXT: store i8 2, ptr [[P2]], align 1
+; CHECK-NEXT: call void @external(ptr [[P]])
; CHECK-NEXT: ret void
;
+ %P = alloca i64
%Y = add i8 %X, 1 ;; Dead, because the only use (the store) is dead.
%P2 = getelementptr i8, ptr %P, i32 2
store i8 %Y, ptr %P2 ;; Not read by lifetime.end, should be removed.
call void @llvm.lifetime.end.p0(i64 1, ptr %P)
store i8 2, ptr %P2
+ call void @external(ptr %P)
ret void
}
-define void @test3a(ptr %P, i8 %X) {
+define void @test3a(i8 %X) {
; CHECK-LABEL: @test3a(
-; CHECK-NEXT: call void @llvm.lifetime.end.p0(i64 10, ptr [[P:%.*]])
+; CHECK-NEXT: [[P:%.*]] = alloca i64, align 8
+; CHECK-NEXT: call void @llvm.lifetime.end.p0(i64 10, ptr [[P]])
; CHECK-NEXT: ret void
;
+ %P = alloca i64
%Y = add i8 %X, 1 ;; Dead, because the only use (the store) is dead.
%P2 = getelementptr i8, ptr %P, i32 2
diff --git a/llvm/test/Analysis/CallGraph/ignore-assumelike-calls.ll b/llvm/test/Analysis/CallGraph/ignore-assumelike-calls.ll
index 658d73804c174..1c9d20193869e 100644
--- a/llvm/test/Analysis/CallGraph/ignore-assumelike-calls.ll
+++ b/llvm/test/Analysis/CallGraph/ignore-assumelike-calls.ll
@@ -10,7 +10,7 @@
; CHECK-EMPTY:
; CHECK-NEXT: Call graph node for function: 'bitcast_only'<<{{.*}}>> #uses=0
; CHECK-EMPTY:
-; CHECK-NEXT: Call graph node for function: 'llvm.lifetime.start.p0'<<{{.*}}>> #uses=3
+; CHECK-NEXT: Call graph node for function: 'llvm.lifetime.start.p0'<<{{.*}}>> #uses=2
; CHECK-EMPTY:
; CHECK-NEXT: Call graph node for function: 'llvm.memset.p0.i64'<<{{.*}}>> #uses=2
; CHECK-EMPTY:
@@ -25,18 +25,11 @@
; CHECK-NEXT: Call graph node for function: 'used_by_lifetime'<<{{.*}}>> #uses=0
; CHECK-NEXT: CS<{{.*}}> calls function 'llvm.lifetime.start.p0'
; CHECK-EMPTY:
-; CHECK-NEXT: Call graph node for function: 'used_by_lifetime_cast'<<{{.*}}>> #uses=0
-; CHECK-NEXT: CS<{{.*}}> calls function 'llvm.lifetime.start.p0'
-; CHECK-EMPTY:
define internal void @used_by_lifetime() {
entry:
- call void @llvm.lifetime.start.p0(i64 4, ptr @used_by_lifetime)
- ret void
-}
-
-define internal void @used_by_lifetime_cast() addrspace(1) {
- call void @llvm.lifetime.start.p0(i64 4, ptr addrspacecast (ptr addrspace(1) @used_by_lifetime_cast to ptr))
+ %a = alloca i8
+ call void @llvm.lifetime.start.p0(i64 4, ptr %a)
ret void
}
diff --git a/llvm/test/Analysis/CostModel/X86/free-intrinsics.ll b/llvm/test/Analysis/CostModel/X86/free-intrinsics.ll
index a8c5c43c3a9f8..3a54428bd8291 100644
--- a/llvm/test/Analysis/CostModel/X86/free-intrinsics.ll
+++ b/llvm/test/Analysis/CostModel/X86/free-intrinsics.ll
@@ -4,6 +4,7 @@
define i32 @trivially_free() {
; CHECK-SIZE-LABEL: 'trivially_free'
+; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %alloca = alloca i8, align 1
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a0 = call i32 @llvm.annotation.i32.p0(i32 undef, ptr undef, ptr undef, i32 undef)
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.assume(i1 undef)
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.experimental.noalias.scope.decl(metadata !3)
@@ -13,14 +14,15 @@ define i32 @trivially_free() {
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a2 = call ptr @llvm.launder.invariant.group.p0(ptr undef)
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a3 = call ptr @llvm.strip.invariant.group.p0(ptr undef)
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a4 = call i1 @llvm.is.constant.i32(i32 undef)
-; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.lifetime.start.p0(i64 1, ptr undef)
-; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.lifetime.end.p0(i64 1, ptr undef)
+; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.lifetime.start.p0(i64 1, ptr %alloca)
+; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.lifetime.end.p0(i64 1, ptr %alloca)
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a5 = call i64 @llvm.objectsize.i64.p0(ptr undef, i1 true, i1 true, i1 true)
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a6 = call ptr @llvm.ptr.annotation.p0.p0(ptr undef, ptr undef, ptr undef, i32 undef, ptr undef)
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.var.annotation.p0.p0(ptr undef, ptr undef, ptr undef, i32 undef, ptr undef)
; CHECK-SIZE-NEXT: Cost Model: Found an estimated cost of 1 for instruction: ret i32 undef
;
; CHECK-THROUGHPUT-LABEL: 'trivially_free'
+; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %alloca = alloca i8, align 1
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a0 = call i32 @llvm.annotation.i32.p0(i32 undef, ptr undef, ptr undef, i32 undef)
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.assume(i1 undef)
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.experimental.noalias.scope.decl(metadata !3)
@@ -30,13 +32,14 @@ define i32 @trivially_free() {
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a2 = call ptr @llvm.launder.invariant.group.p0(ptr undef)
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a3 = call ptr @llvm.strip.invariant.group.p0(ptr undef)
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a4 = call i1 @llvm.is.constant.i32(i32 undef)
-; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.lifetime.start.p0(i64 1, ptr undef)
-; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.lifetime.end.p0(i64 1, ptr undef)
+; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.lifetime.start.p0(i64 1, ptr %alloca)
+; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.lifetime.end.p0(i64 1, ptr %alloca)
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a5 = call i64 @llvm.objectsize.i64.p0(ptr undef, i1 true, i1 true, i1 true)
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: %a6 = call ptr @llvm.ptr.annotation.p0.p0(ptr undef, ptr undef, ptr undef, i32 undef, ptr undef)
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: call void @llvm.var.annotation.p0.p0(ptr undef, ptr undef, ptr undef, i32 undef, ptr undef)
; CHECK-THROUGHPUT-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret i32 undef
;
+ %alloca = alloca i8
%a0 = call i32 @llvm.annotation.i32(i32 undef, ptr undef, ptr undef, i32 undef)
call void @llvm.assume(i1 undef)
call void @llvm.experimental.noalias.scope.decl(metadata !...
[truncated]
|
efriedma-quic
approved these changes
Jul 18, 2025
There was a problem hiding this comment.
LGTM
There's possibly some underlying need for "lifetimes" with different rules. Like, if you want to implement malloc. But allocas are special because the backend handles allocation/deallocation, so I think it makes sense to have dedicated intrinsics specifically for alloca.
nikic
added a commit
that referenced
this pull request
Jul 23, 2025
) When materializing a function, we'd upgrade all calls to all upgraded intrinsics. However, this would operate on all calls to the intrinsic (including previously materialized ones), which leads to quadratic complexity. Instead, only upgrade the calls that are in the materialized function. This fixes a compile-time regression introduced by #149310.
nikic
added a commit
that referenced
this pull request
Jul 23, 2025
After #149310 this is guaranteed to be an alloca.
nikic
added a commit
to nikic/llvm-project
that referenced
this pull request
Jul 23, 2025
Now that llvm#149310 has restricted lifetime intrinsics to only work on allocas, we can also drop the explicit size argument. Instead, the size is implied by the alloca. This removes the ability to only mark a prefix of an alloca alive/dead. We never used that capability, so we should remove the need to handle that possibility everywhere (many key places, including stack coloring, did not actually respect this).
nikic
added a commit
to nikic/llvm-project
that referenced
this pull request
Jul 24, 2025
Now that llvm#149310 has restricted lifetime intrinsics to only work on allocas, we can also drop the explicit size argument. Instead, the size is implied by the alloca. This removes the ability to only mark a prefix of an alloca alive/dead. We never used that capability, so we should remove the need to handle that possibility everywhere (many key places, including stack coloring, did not actually respect this).
nikic
added a commit
to nikic/llvm-project
that referenced
this pull request
Jul 25, 2025
Now that llvm#149310 has restricted lifetime intrinsics to only work on allocas, we can also drop the explicit size argument. Instead, the size is implied by the alloca. This removes the ability to only mark a prefix of an alloca alive/dead. We never used that capability, so we should remove the need to handle that possibility everywhere (many key places, including stack coloring, did not actually respect this).
nikic
added a commit
to nikic/llvm-project
that referenced
this pull request
Jul 25, 2025
Now that llvm#149310 has restricted lifetime intrinsics to only work on allocas, we can also drop the explicit size argument. Instead, the size is implied by the alloca. This removes the ability to only mark a prefix of an alloca alive/dead. We never used that capability, so we should remove the need to handle that possibility everywhere (many key places, including stack coloring, did not actually respect this).
|
Enzyme has use cases where we use the lifetime intrinsic on non-allocas [specifically malloc for tracking lifetime]. Is there expected to be a replacement lifetime intrinsic for non-allocas? |
|
I don't know if anyone is currently planning to implement anything, but I have some ideas. One approach is to introduce an intrinsic Another option is a pair of intrinsics: |
This is the design I had in mind. A benefit of divorcing this from any lifetime related concepts is that we could use this to solve the loop DSE problem (have LIR insert this intrinsic before loops, allowing DSE to work as usual). A minor variant on this would be to not actually use a separate intrinsic for this, but just use |
|
If the design for these other use cases hasn't been finalized, is it reasonable to revert the assertion until a workaround is available? |
|
@wsmoses After looking at what Enzyme does, I think you just need to drop creation of those lifetime intrinsics. You are calling lifetime.start directly after malloc and lifetime.end directly before free, which does not do anything useful. |
mahesh-attarde
pushed a commit
to mahesh-attarde/llvm-project
that referenced
this pull request
Jul 28, 2025
lifetime.start and lifetime.end are primarily intended for use on allocas, to enable stack coloring and other liveness optimizations. This is necessary because all (static) allocas are hoisted into the entry block, so lifetime markers are the only way to convey the actual lifetimes. However, lifetime.start and lifetime.end are currently *allowed* to be used on non-alloca pointers. We don't actually do this in practice, but just the mere fact that this is possible breaks the core purpose of the lifetime markers, which is stack coloring of allocas. Stack coloring can only work correctly if all lifetime markers for an alloca are analyzable. * If a lifetime marker may operate on multiple allocas via a select/phi, we don't know which lifetime actually starts/ends and handle it incorrectly (llvm#104776). * Stack coloring operates on the assumption that all lifetime markers are visible, and not, for example, hidden behind a function call or escaped pointer. It's not possible to change this, as part of the purpose of lifetime markers is that they work even in the presence of escaped pointers, where simple use analysis is insufficient. I don't think there is any way to have coherent semantics for lifetime markers on allocas, while also permitting them on arbitrary pointer values. This PR restricts lifetimes to operate on allocas only. As a followup, I will also drop the size argument, which is superfluous if we always operate on an alloca. (This change also renders various code handling lifetime markers on non-alloca dead. I plan to clean up that kind of code after dropping the size argument as well.) In practice, I've only found a few places that currently produce lifetimes on non-allocas: * CoroEarly replaces the promise alloca with the result of an intrinsic, which will later be replaced back with an alloca. I think this is the only place where there is some legitimate loss of functionality, but I don't think this is particularly important (I don't think we'd expect the promise in a coroutine to admit useful lifetime optimization.) * SafeStack moves unsafe allocas onto a separate frame. We can safely drop lifetimes here, as SafeStack performs its own stack coloring. * Similar for AddressSanitizer, it also moves allocas into separate memory. * LSR sometimes replaces the lifetime argument with a GEP chain of the alloca (where the offsets ultimately cancel out). This is just unnecessary. (Fixed separately in llvm#149492.) * InferAddrSpaces sometimes makes lifetimes operate on an addrspacecast of an alloca. I don't think this is necessary.
mahesh-attarde
pushed a commit
to mahesh-attarde/llvm-project
that referenced
this pull request
Jul 28, 2025
…XIL version is lower than 1.6 (llvm#147432)" (llvm#149874) This PR reverts commit d47c126 (corresponding to PR llvm#147432) to fix a build failure caused by llvm#149310
mahesh-attarde
pushed a commit
to mahesh-attarde/llvm-project
that referenced
this pull request
Jul 28, 2025
Reverts llvm#148003 to fix a DirectX backend build breakage due to llvm#149310
mahesh-attarde
pushed a commit
to mahesh-attarde/llvm-project
that referenced
this pull request
Jul 28, 2025
This was always undesirable, and after llvm#149310 it is illegal and will result in a verifier error. Fix this by moving SimplifyCFG's check for this into canReplaceOperandWithVariable(), so it's shared with GVNSink.
mahesh-attarde
pushed a commit
to mahesh-attarde/llvm-project
that referenced
this pull request
Jul 28, 2025
…149838) After llvm#149310 we are guaranteed that the argument is an alloca, so we don't need to look at underlying objects (which was not a correct thing to do anyway). This also drops the offset argument for lifetime nodes in SDAG. The offset is fixed to zero now. (Peculiarly, while SDAG pretended to have an offset, it just gets silently dropped during selection.)
mahesh-attarde
pushed a commit
to mahesh-attarde/llvm-project
that referenced
this pull request
Jul 28, 2025
…lvm#149994) After llvm#149310 the pointer argument of lifetime.start/lifetime.end is guaranteed to be an alloca, so we don't need to go through findAllocaForValue() anymore, and don't have to have special handling for the case where it fails.
mahesh-attarde
pushed a commit
to mahesh-attarde/llvm-project
that referenced
this pull request
Jul 28, 2025
…#150032) When materializing a function, we'd upgrade all calls to all upgraded intrinsics. However, this would operate on all calls to the intrinsic (including previously materialized ones), which leads to quadratic complexity. Instead, only upgrade the calls that are in the materialized function. This fixes a compile-time regression introduced by llvm#149310.
mahesh-attarde
pushed a commit
to mahesh-attarde/llvm-project
that referenced
this pull request
Jul 28, 2025
After llvm#149310 this is guaranteed to be an alloca.
nikic
added a commit
to nikic/llvm-project
that referenced
this pull request
Jul 28, 2025
After llvm#149310 lifetime intrinsics require an alloca argument, an invariant that this pass can break. I've fixed this in two ways: * First, move static allocas into the entry block. Currently, the way the pass splits the entry block makes all allocas dynamic, which I assume was not actually intended. This will avoid unnecessary SSA reconstruction for allocas as well, and thus avoid the problem. * If this fails (for dynamic allocas) drop all lifetime intrinsics if any one of them would require a rewrite during SSA reconstruction. Fixes llvm#150498.
nikic
added a commit
that referenced
this pull request
Jul 29, 2025
After #149310 lifetime intrinsics require an alloca argument, an invariant that this pass can break. I've fixed this in two ways: * First, move static allocas into the entry block. Currently, the way the pass splits the entry block makes all allocas dynamic, which I assume was not actually intended. This will avoid unnecessary SSA reconstruction for allocas as well, and thus avoid the problem. * If this fails (for dynamic allocas) drop all lifetime intrinsics if any one of them would require a rewrite during SSA reconstruction. Fixes #150498.
nikic
added a commit
to nikic/llvm-project
that referenced
this pull request
Jul 29, 2025
This slightly relaxes the invariant established in llvm#149310, by also allowing the lifetime argument to be poison. This is to support the typical pattern of RAUWing with poison when removing an instruction. It's worth noting that this does not require any conservative assumptions, lifetimes with poison arguments can simply be skipped. Fixes llvm#151119.
nikic
added a commit
to nikic/llvm-project
that referenced
this pull request
Jul 29, 2025
Now that llvm#149310 has restricted lifetime intrinsics to only work on allocas, we can also drop the explicit size argument. Instead, the size is implied by the alloca. This removes the ability to only mark a prefix of an alloca alive/dead. We never used that capability, so we should remove the need to handle that possibility everywhere (many key places, including stack coloring, did not actually respect this).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
lifetime.start and lifetime.end are primarily intended for use on allocas, to enable stack coloring and other liveness optimizations. This is necessary because all (static) allocas are hoisted into the entry block, so lifetime markers are the only way to convey the actual lifetimes.
However, lifetime.start and lifetime.end are currently allowed to be used on non-alloca pointers. We don't actually do this in practice, but just the mere fact that this is possible breaks the core purpose of the lifetime markers, which is stack coloring of allocas. Stack coloring can only work correctly if all lifetime markers for an alloca are analyzable.
I don't think there is any way to have coherent semantics for lifetime markers on allocas, while also permitting them on arbitrary pointer values.
This PR restricts lifetimes to operate on allocas only. As a followup, I will also drop the size argument, which is superfluous if we always operate on an alloca. (This change also renders various code handling lifetime markers on non-alloca dead. I plan to clean up that kind of code after dropping the size argument as well.)
In practice, I've only found a few places that currently produce lifetimes on non-allocas: