[v24.3.x] tx/group compaction fixes #24688
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(cherry picked from commit 8947848)
(cherry picked from commit cdd274a)
(cherry picked from commit 24c8e89)
This is unsafe because it does not do any required checks to see if a particular transaction is in progress and is a candidate for abort. For example if a transaction is committed by the coordinator and pending commit on the group, using this escape hatch to abort the transaction can cause correctness issues. To be used with caution as an escape hatch for aborting transactions that the group has lost track of are ok to be aborted. This situation usually is indicative of a bug in the transaction implementation. (cherry picked from commit 8c5ecca)
(cherry picked from commit 3736f00)
(cherry picked from commit 835f3fc)
Consider group_metadata to determine if a group transaction should be considered open. Eg: if a group if tombstoned, any transaction corresponding to the group is ignored. This invariant is also held in the actual group stm to ensure groups are not tombstoned before any pending transactions are cleaned up (cherry picked from commit 9eee632)
(cherry picked from commit f7191ad)
(cherry picked from commit 6ca81b4)
Retry command for Build#60285please wait until all jobs are finished before running the slash command |
CI test resultstest results on build#60285
test results on build#60301
|
(cherry picked from commit 6fc62bb)
This will result in hanging transactions and subsequent blocking of compaction. (cherry picked from commit 2b79687)
(cherry picked from commit ac22041)
If a group got tombstoned all the producers to that group should be ignored. The current logic is incorrectly recovering producers and loading them up to expire later. (cherry picked from commit 7c8d633)
(cherry picked from commit c7f953e)
(cherry picked from commit 9958ca6)
.. for a given partition, to be hooked up with REST API in the next commit. (cherry picked from commit 6efd325)
(cherry picked from commit 23c8e29)
/v1/debug/producers/{namespace}/{topic}/{partition}
.. includes low level debug information about producers for
idempotency/transactional state.
(cherry picked from commit 70e36eb)
.. in this case the state machine proceeds on to applying from the log. (cherry picked from commit c833f50)
Bumps the supported snapshot version so the existing snapshots are invalidated as they may contain stale max_collectible_offset. This forces the stm to reconstruct the state form the log and recompute correct max_collectible_offset. (cherry picked from commit 0051463)
bharathv
force-pushed
the
243x-comp-fix
branch
from
69057e7 to
2ca2efc
Compare
Retry command for Build#60301please wait until all jobs are finished before running the slash command |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport of PR #24637
Fixes #24684
Backports Required
Release Notes
Bug Fixes