[VectorUtils] Add llvm::scaleShuffleMaskElts wrapper for narrowShuffleMaskElts/widenShuffleMaskElts #96646
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…eMaskElts/widenShuffleMaskElts Using the target number of vector elements, scaleShuffleMaskElts will try to use narrowShuffleMaskElts/widenShuffleMaskElts to scale the shuffle mask accordingly. Working on llvm#58895 I didn't want to create yet another case where we have to handle both re-scaling cases.
llvmbot
added
llvm:analysis
|
@llvm/pr-subscribers-llvm-analysis Author: Simon Pilgrim (RKSimon) ChangesUsing the target number of vector elements, scaleShuffleMaskElts will try to use narrowShuffleMaskElts/widenShuffleMaskElts to scale the shuffle mask accordingly. Working on #58895 I didn't want to create yet another case where we have to handle both re-scaling cases. Full diff: https://github.com/llvm/llvm-project/pull/96646.diff 3 Files Affected:
diff --git a/llvm/include/llvm/Analysis/VectorUtils.h b/llvm/include/llvm/Analysis/VectorUtils.h
index 7a740d1206f7c..db3b62807cc67 100644
--- a/llvm/include/llvm/Analysis/VectorUtils.h
+++ b/llvm/include/llvm/Analysis/VectorUtils.h
@@ -224,6 +224,13 @@ void narrowShuffleMaskElts(int Scale, ArrayRef<int> Mask,
bool widenShuffleMaskElts(int Scale, ArrayRef<int> Mask,
SmallVectorImpl<int> &ScaledMask);
+/// Attempt to narrow/widen the \p Mask shuffle mask to the \p NumDstElts target
+/// width. Internally this will call narrowShuffleMaskElts/widenShuffleMaskElts.
+/// This will fail unless NumDstElts is a multiple of Mask.size (or vice-versa).
+/// Returns false on failure, and ScaledMask will be in an undefined state.
+bool scaleShuffleMaskElts(unsigned NumDstElts, ArrayRef<int> Mask,
+ SmallVectorImpl<int> &ScaledMask);
+
/// Repetitively apply `widenShuffleMaskElts()` for as long as it succeeds,
/// to get the shuffle mask with widest possible elements.
void getShuffleMaskWithWidestElts(ArrayRef<int> Mask,
diff --git a/llvm/lib/Analysis/VectorUtils.cpp b/llvm/lib/Analysis/VectorUtils.cpp
index 6e183bbe5bff1..856f34ee1d5da 100644
--- a/llvm/lib/Analysis/VectorUtils.cpp
+++ b/llvm/lib/Analysis/VectorUtils.cpp
@@ -418,6 +418,31 @@ bool llvm::widenShuffleMaskElts(int Scale, ArrayRef<int> Mask,
return true;
}
+bool llvm::scaleShuffleMaskElts(unsigned NumDstElts, ArrayRef<int> Mask,
+ SmallVectorImpl<int> &ScaledMask) {
+ unsigned NumSrcElts = Mask.size();
+ assert(NumSrcElts > 0 && NumDstElts > 0 && "Unexpected scaling factor");
+
+ // Fast-path: if no scaling, then it is just a copy.
+ if (NumSrcElts == NumDstElts) {
+ ScaledMask.assign(Mask.begin(), Mask.end());
+ return true;
+ }
+
+ // We can't find a whole scale factor - bail out.
+ if ((NumSrcElts % NumDstElts) != 0 && (NumDstElts % NumSrcElts) != 0)
+ return false;
+
+ if (NumSrcElts > NumDstElts) {
+ int Scale = NumSrcElts / NumDstElts;
+ return widenShuffleMaskElts(Scale, Mask, ScaledMask);
+ }
+
+ int Scale = NumDstElts / NumSrcElts;
+ narrowShuffleMaskElts(Scale, Mask, ScaledMask);
+ return true;
+}
+
void llvm::getShuffleMaskWithWidestElts(ArrayRef<int> Mask,
SmallVectorImpl<int> &ScaledMask) {
std::array<SmallVector<int, 16>, 2> TmpMasks;
diff --git a/llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp b/llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp
index ebc2930d33d26..3de56a4038039 100644
--- a/llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp
+++ b/llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp
@@ -2837,15 +2837,7 @@ Instruction *InstCombinerImpl::visitShuffleVectorInst(ShuffleVectorInst &SVI) {
auto *XType = cast<FixedVectorType>(X->getType());
unsigned XNumElts = XType->getNumElements();
SmallVector<int, 16> ScaledMask;
- if (XNumElts >= VWidth) {
- assert(XNumElts % VWidth == 0 && "Unexpected vector bitcast");
- narrowShuffleMaskElts(XNumElts / VWidth, Mask, ScaledMask);
- } else {
- assert(VWidth % XNumElts == 0 && "Unexpected vector bitcast");
- if (!widenShuffleMaskElts(VWidth / XNumElts, Mask, ScaledMask))
- ScaledMask.clear();
- }
- if (!ScaledMask.empty()) {
+ if (scaleShuffleMaskElts(XNumElts, Mask, ScaledMask)) {
// If the shuffled source vector simplifies, cast that value to this
// shuffle's type.
if (auto *V = simplifyShuffleVectorInst(X, UndefValue::get(XType),
|
|
@llvm/pr-subscribers-llvm-transforms Author: Simon Pilgrim (RKSimon) ChangesUsing the target number of vector elements, scaleShuffleMaskElts will try to use narrowShuffleMaskElts/widenShuffleMaskElts to scale the shuffle mask accordingly. Working on #58895 I didn't want to create yet another case where we have to handle both re-scaling cases. Full diff: https://github.com/llvm/llvm-project/pull/96646.diff 3 Files Affected:
diff --git a/llvm/include/llvm/Analysis/VectorUtils.h b/llvm/include/llvm/Analysis/VectorUtils.h
index 7a740d1206f7c..db3b62807cc67 100644
--- a/llvm/include/llvm/Analysis/VectorUtils.h
+++ b/llvm/include/llvm/Analysis/VectorUtils.h
@@ -224,6 +224,13 @@ void narrowShuffleMaskElts(int Scale, ArrayRef<int> Mask,
bool widenShuffleMaskElts(int Scale, ArrayRef<int> Mask,
SmallVectorImpl<int> &ScaledMask);
+/// Attempt to narrow/widen the \p Mask shuffle mask to the \p NumDstElts target
+/// width. Internally this will call narrowShuffleMaskElts/widenShuffleMaskElts.
+/// This will fail unless NumDstElts is a multiple of Mask.size (or vice-versa).
+/// Returns false on failure, and ScaledMask will be in an undefined state.
+bool scaleShuffleMaskElts(unsigned NumDstElts, ArrayRef<int> Mask,
+ SmallVectorImpl<int> &ScaledMask);
+
/// Repetitively apply `widenShuffleMaskElts()` for as long as it succeeds,
/// to get the shuffle mask with widest possible elements.
void getShuffleMaskWithWidestElts(ArrayRef<int> Mask,
diff --git a/llvm/lib/Analysis/VectorUtils.cpp b/llvm/lib/Analysis/VectorUtils.cpp
index 6e183bbe5bff1..856f34ee1d5da 100644
--- a/llvm/lib/Analysis/VectorUtils.cpp
+++ b/llvm/lib/Analysis/VectorUtils.cpp
@@ -418,6 +418,31 @@ bool llvm::widenShuffleMaskElts(int Scale, ArrayRef<int> Mask,
return true;
}
+bool llvm::scaleShuffleMaskElts(unsigned NumDstElts, ArrayRef<int> Mask,
+ SmallVectorImpl<int> &ScaledMask) {
+ unsigned NumSrcElts = Mask.size();
+ assert(NumSrcElts > 0 && NumDstElts > 0 && "Unexpected scaling factor");
+
+ // Fast-path: if no scaling, then it is just a copy.
+ if (NumSrcElts == NumDstElts) {
+ ScaledMask.assign(Mask.begin(), Mask.end());
+ return true;
+ }
+
+ // We can't find a whole scale factor - bail out.
+ if ((NumSrcElts % NumDstElts) != 0 && (NumDstElts % NumSrcElts) != 0)
+ return false;
+
+ if (NumSrcElts > NumDstElts) {
+ int Scale = NumSrcElts / NumDstElts;
+ return widenShuffleMaskElts(Scale, Mask, ScaledMask);
+ }
+
+ int Scale = NumDstElts / NumSrcElts;
+ narrowShuffleMaskElts(Scale, Mask, ScaledMask);
+ return true;
+}
+
void llvm::getShuffleMaskWithWidestElts(ArrayRef<int> Mask,
SmallVectorImpl<int> &ScaledMask) {
std::array<SmallVector<int, 16>, 2> TmpMasks;
diff --git a/llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp b/llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp
index ebc2930d33d26..3de56a4038039 100644
--- a/llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp
+++ b/llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp
@@ -2837,15 +2837,7 @@ Instruction *InstCombinerImpl::visitShuffleVectorInst(ShuffleVectorInst &SVI) {
auto *XType = cast<FixedVectorType>(X->getType());
unsigned XNumElts = XType->getNumElements();
SmallVector<int, 16> ScaledMask;
- if (XNumElts >= VWidth) {
- assert(XNumElts % VWidth == 0 && "Unexpected vector bitcast");
- narrowShuffleMaskElts(XNumElts / VWidth, Mask, ScaledMask);
- } else {
- assert(VWidth % XNumElts == 0 && "Unexpected vector bitcast");
- if (!widenShuffleMaskElts(VWidth / XNumElts, Mask, ScaledMask))
- ScaledMask.clear();
- }
- if (!ScaledMask.empty()) {
+ if (scaleShuffleMaskElts(XNumElts, Mask, ScaledMask)) {
// If the shuffled source vector simplifies, cast that value to this
// shuffle's type.
if (auto *V = simplifyShuffleVectorInst(X, UndefValue::get(XType),
|
dtcxzyw
reviewed
Jun 25, 2024
llvm/lib/Analysis/VectorUtils.cpp
Outdated
| } | ||
|
|
||
| // We can't find a whole scale factor - bail out. | ||
| if ((NumSrcElts % NumDstElts) != 0 && (NumDstElts % NumSrcElts) != 0) |
There was a problem hiding this comment.
I'd left this as a bail out in case future uses won't guarantee this, but I've updated it to an assertion for now.
dtcxzyw
approved these changes
Jun 25, 2024
lravenclaw
pushed a commit
to lravenclaw/llvm-project
that referenced
this pull request
Jul 3, 2024
…eMaskElts/widenShuffleMaskElts, NFC. (llvm#96646) Using the target number of vector elements, scaleShuffleMaskElts will try to use narrowShuffleMaskElts/widenShuffleMaskElts to scale the shuffle mask accordingly. Working on llvm#58895 I didn't want to create yet another case where we have to handle both re-scaling cases.
AlexisPerry
pushed a commit
to llvm-project-tlp/llvm-project
that referenced
this pull request
Jul 9, 2024
…eMaskElts/widenShuffleMaskElts, NFC. (llvm#96646) Using the target number of vector elements, scaleShuffleMaskElts will try to use narrowShuffleMaskElts/widenShuffleMaskElts to scale the shuffle mask accordingly. Working on llvm#58895 I didn't want to create yet another case where we have to handle both re-scaling cases.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Using the target number of vector elements, scaleShuffleMaskElts will try to use narrowShuffleMaskElts/widenShuffleMaskElts to scale the shuffle mask accordingly.
Working on #58895 I didn't want to create yet another case where we have to handle both re-scaling cases.