[LI-HOTFIX] Support full LeaderAndISR through LiCombinedControl requests #427
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
TICKET = LIKAFKA-49560 LI_DESCRIPTION = As described in the ticket, we found that the LiCombinedControl requests can be disabled for newly added brokers. The newly added brokers experience the following bebavior 1. the LiCombinedContrel requests are enabled when the broker starts up and host 0 replicas 2. the LiCombinedControl requests are disabled when some replicas are assigned to the broker 3. the LiCombinedControl requests can only be re-enabled after the broker is restarted The reason for the problem in step 2 is that once the LiCombinedControl request is enabled and a full LeaderAndISR request needs to be sent, it will try to merge request while doesn't honor the full request type. As a result, brokers receive the LeaderAndISR as part of the LiCombinedControl, and treat it as an incremental LeaderAndISR instead of a full request. This PR tries to address the problem by having full support of the LeaderAndISR request type within LiCombinedControl. EXIT_CRITERIA = The same as the LiCombinedControl request.
e8cf30e to
80cfa96
Compare
groelofs
approved these changes
Jan 13, 2023
groelofs
left a comment
There was a problem hiding this comment.
LGTM, just a question about how much latitude we have on merging. From the description, it seemed like fixing the broker-handling half (i.e., making them aware of full vs. delta LAIR within LCCR) would suffice as a fix. It's not clear to me what the "stop merging at first full LAIR" part adds to the equation.
core/src/main/scala/kafka/controller/ControllerRequestMerger.scala
Outdated
Show resolved
Hide resolved
lmr3796
approved these changes
Jan 13, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TICKET = LIKAFKA-49560
LI_DESCRIPTION =
As described in the ticket, we found that the LiCombinedControl requests can be disabled for newly
added brokers. The newly added brokers experience the following behavior
The reason for the problem in step 2 is that once the LiCombinedControl request is enabled
and a full LeaderAndISR request needs to be sent, it will try to merge the request while
doesn't honor the full request type. As a result, brokers receive the LeaderAndISR as
part of the LiCombinedControl, and treat it as an incremental LeaderAndISR instead of a full
request.
This PR tries to address the problem by having the full support of the LeaderAndISR request type
within LiCombinedControl.
EXIT_CRITERIA = The same as the LiCombinedControl request.