[LLVMGPUVectorDistribute] Add support for inter-subgroup multi_reduction #19596
+390
−159
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Im putting this to draft as I have not yet numerically verified these changes. This PR basically implements approach 3 of https://hackmd.io/bXwT715pQU-8rrS78LKCbw |
manupak
commented
Jan 3, 2025
| @@ -235,11 +235,11 @@ static void applyVectorDistribution(Operation *root, | |||
| applicator.applyDefaultCostModel(); | |||
|
|
|||
| // Collect all the operations to be distributed. | |||
| std::deque<Operation *> worklist; | |||
| llvm::SmallDenseSet<Operation *> worklist; | |||
There was a problem hiding this comment.
I think vector distribution insensitive to work ordering. Am I wrong here?
Why this here now ?
I had to do this because re-distribution listener will keep adding ops to work list even if we do a operand change. Then we end up with same operation in the worklist multiple times. If folks have a better solution, Im all ears :)
manupak
force-pushed
the
inter-sg-reduce
branch
2 times, most recently
from
151cc55 to
2029ed7
Compare
This commit adds support for distribute multi_reductions where the reduction dimension(s) is/are distributed across subgroups. We perform the existing reduction distribution, however, we are left with partial reductions accross subgroups. Thereafter, we insert tranfer_write / transfer_read to shared memory to achieve a layout change where we re-distribute reduction subgroup tiles into element tile. Finally, we do another multi_reduction to complete the reduction. Signed-off-by: Manupa Karunaratne <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit adds support for distribute multi_reductions where the reduction dimension(s) is/are distributed across subgroups.
We perform the existing reduction distribution, however, we are left with partial reductions accross subgroups.
Thereafter, we insert tranfer_write / transfer_read to shared memory to achieve a layout change where
we re-distribute reduction subgroup tiles into element tile. Finally, we do another multi_reduction to complete the reduction.
closes: #19578