Skip to content

Conversation

@likg227
Copy link
Contributor

@likg227 likg227 commented Mar 17, 2022

9## What's changed and what's your intention?

PLEASE DO NOT LEAVE THIS EMPTY !!!

Please explain IN DETAIL what the changes are in this PR and why they are needed:

  • Summarize your change (mandatory)
  • How does this PR work? Need a brief introduction for the changed logic (optional)
  • Describe clearly one logical change and avoid lazy messages (optional)
  • Describe any limitations of the current code (optional)

In this pr, I TRY to add four physical aggs, but I'm not sure I've got the right solution.

Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests

Refer to a related PR or issue link (optional)

close #929
close #943

@likg227 likg227 requested review from fuyufjh and xiangjinwu March 17, 2022 08:12
@github-actions github-actions bot added the type/feature Type: New feature. label Mar 17, 2022
fuyufjh
fuyufjh previously approved these changes Mar 17, 2022
Copy link
Collaborator

@fuyufjh fuyufjh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment on lines 62 to 66
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's not correct. The input need to be distributed by group key

@fuyufjh fuyufjh dismissed their stale review March 17, 2022 08:24

see comments above

Comment on lines 58 to 61
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly, SimpleAgg needs input to be distributed in singoton

Comment on lines +21 to +29
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if select has group by but no agg function within select items?

create table t (a char(10), b int, c bool);
select t.a from t group by t.c;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This query is invalid....

postgres=# select t.a from t group by t.c;
ERROR: column "t.a" must appear in the GROUP BY clause or be used in an aggregate function
LINE 1: select t.a from t group by t.c;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, so are we able to prompt error on this sql?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we can.

dev=> select v1 from t group by v2;
ERROR: Invalid input syntax: column must appear in the GROUP BY clause or be used in an aggregate function

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can add some error cases in the planner test, although they are already covered by unit tests of LogicalAgg::create.

Comment on lines 35 to 38
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's be more compact and follow expr::AggCall:

            let mut builder = f.debug_tuple(&format!("{:?}", self.agg_kind));
            self.inputs.iter().for_each(|child| {
                builder.field(child);
            });
            builder.finish()

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: groups before aggs

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: to_stream

Copy link
Contributor

@xiangjinwu xiangjinwu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest lgtm

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The output distribution of HashAgg and SimpleAgg are also by-groups and Single respectively.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to logical, groups before aggs.

Comment on lines +21 to +29
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can add some error cases in the planner test, although they are already covered by unit tests of LogicalAgg::create.

@codecov
Copy link

codecov bot commented Mar 17, 2022

Codecov Report

Merging #1026 (c7e2b22) into main (95b9d6f) will increase coverage by 0.11%.
The diff coverage is 94.16%.

❗ Current head c7e2b22 differs from pull request most recent head 2b132e8. Consider uploading reports for the commit 2b132e8 to get more accurate results

@@             Coverage Diff              @@
##               main    #1026      +/-   ##
============================================
+ Coverage     71.39%   71.50%   +0.11%     
  Complexity     2766     2766              
============================================
  Files           978      982       +4     
  Lines         57782    57936     +154     
  Branches       1790     1790              
============================================
+ Hits          41254    41430     +176     
+ Misses        15637    15615      -22     
  Partials        891      891              
Flag Coverage Δ
java 61.03% <ø> (ø)
rust 75.45% <94.16%> (+0.14%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
rust/frontend/src/optimizer/plan_node/mod.rs 57.37% <50.00%> (-0.52%) ⬇️
...ntend/src/optimizer/plan_node/stream_simple_agg.rs 87.50% <87.50%> (ø)
...rontend/src/optimizer/plan_node/stream_hash_agg.rs 89.47% <89.47%> (ø)
...st/frontend/src/optimizer/plan_node/logical_agg.rs 96.88% <93.75%> (+2.20%) ⬆️
rust/frontend/src/binder/select.rs 98.36% <100.00%> (+0.28%) ⬆️
rust/frontend/src/handler/util.rs 78.48% <100.00%> (+0.27%) ⬆️
...frontend/src/optimizer/plan_node/batch_hash_agg.rs 100.00% <100.00%> (ø)
...ontend/src/optimizer/plan_node/batch_simple_agg.rs 100.00% <100.00%> (ø)
rust/frontend/src/planner/select.rs 100.00% <100.00%> (ø)
rust/frontend/test_runner/src/lib.rs 72.91% <100.00%> (+1.04%) ⬆️
... and 20 more

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@likg227 likg227 merged commit b0dbb60 into main Mar 18, 2022
@likg227 likg227 deleted the lkg/plan-agg branch March 18, 2022 02:40
pangzhenzhou pushed a commit that referenced this pull request Mar 28, 2022
* add group_by for BoundSelect.

* plan agg.

* add tests.

* fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type/feature Type: New feature.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

frontend: physical plan nodes for Agg frontend: aggregation function is planned in Project as scalar functions

5 participants