feat(streaming): support union all for streaming query #6397
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## main #6397 +/- ##
==========================================
- Coverage 74.05% 74.00% -0.05%
==========================================
Files 972 975 +3
Lines 158148 158684 +536
==========================================
+ Hits 117115 117436 +321
- Misses 41033 41248 +215
Flags with carried forward coverage won't be shown. Click here to find out more.
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
TennyZhuang
reviewed
Nov 17, 2022
TennyZhuang
reviewed
Nov 17, 2022
|
|
||
| impl<PlanRef: GenericPlanRef> GenericPlanNode for Union<PlanRef> { | ||
| fn schema(&self) -> Schema { | ||
| self.inputs[0].schema().clone() |
Contributor
There was a problem hiding this comment.
Should we add the hidden column here?
Contributor
Author
There was a problem hiding this comment.
Probably not. The input schema already contains the hidden column and we can find them by source_col.
TennyZhuang
reviewed
Nov 17, 2022
TennyZhuang
reviewed
Nov 17, 2022
st1page
reviewed
Nov 17, 2022
Contributor
st1page
left a comment
There was a problem hiding this comment.
Rest LGTM. We can find a better way when we move the pk rewriting only on stream plan 🥵
fuyufjh
approved these changes
Nov 18, 2022
xxchan
pushed a commit
that referenced
this pull request
Nov 19, 2022
* support union all for streaming query * fmt * add stream_dist_plan for union Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
5 tasks
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.
What's changed and what's your intention?
PLEASE DO NOT LEAVE THIS EMPTY !!!
Please explain IN DETAIL what the changes are in this PR and why they are needed:
UNION ALLfor streaming queries.source_colfor rewriting the union operator to identify the record that came from the input source and make it a part of the union pks.UNION ALL, but notUNION. I think all theUNIONshould be transformed intoUNION ALLby optimizer in the later PR to avoid the handcrafted streaming rewriting.Checklist
./risedev check(or alias,./risedev c)Documentation
If your pull request contains user-facing changes, please specify the types of the changes, and create a release note. Otherwise, please feel free to remove this section.
Types of user-facing changes
Please keep the types that apply to your changes, and remove those that do not apply.
Release note
Please create a release note for your changes. In the release note, focus on the impact on users, and mention the environment or conditions where the impact may occur.
Refer to a related PR or issue link (optional)
#6392