Skip to content

feat: introduce rw_worker_nodes, rw_parallel_units and support show clusters command #10656

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Jul 3, 2023
Merged

feat: introduce rw_worker_nodes, rw_parallel_units and support show clusters command #10656

merged 5 commits into from
Jul 3, 2023

Conversation

@yezizp2012
Copy link
Member

@yezizp2012 yezizp2012 commented Jun 29, 2023 •
edited
Loading

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Related #8803.

  1. introduce system tables rw_worker_nodes and rw_parallel_units:
  • rw_worker_nodes
Field Type Description
id Int32 Unique identifier of the worker node
host Varchar Host of the worker node
port Varchar Serving port of the worker node
type Varchar Type of the worker node
state Varchar State of the worker node, should be one of [RUNNING, STARTING]
parallelism Int32 Streaming parallelism of the worker node
is_streaming Boolean Whether the worker node is in streaming mode
is_serving Boolean Whether the worker node is in batch serving mode
is_unschedulable Boolean Whether the worker node is unschedulable for streaming job
  • rw_parallel_units
Field Type Description
id Int32 Unique identifier of the parallel unit
worker_id Int32 Identifier of the worker id that the parallel unit belongs to
  1. support command show clusters:
dev=> show clusters;
      Addr      |  State  |     Parallel Units     | Is Streaming | Is Serving | Is Unschedulable
----------------+---------+------------------------+--------------+------------+------------------
 127.0.0.1:5688 | RUNNING | 3000, 3001, 3002, 3003 | true         | true       | false
(1 row)
  1. change type of columns in rw_sources to array type of varchar and remove column primary_keys.
  2. change type of arg_type_ids in rw_functions to array type of integer.

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
  • All checks passed in ./risedev check (or alias, ./risedev c)

Documentation

  • My PR contains user-facing changes.
Click here for Documentation

Types of user-facing changes

Please keep the types that apply to your changes, and remove the others.

  • SQL commands, functions, and operators

Release note

  1. introduce system tables rw_worker_nodes and rw_parallel_units.
  2. support command SHOW CLUSTERS.
  3. remove primary_keys in rw_sources.

@github-actions github-actions bot added type/feature Type: New feature. user-facing-changes Contains changes that are visible to users labels Jun 29, 2023
@yezizp2012
Copy link
Member Author

yezizp2012 commented Jun 29, 2023

Some examples of a local run:

dev=> select * from rw_worker_nodes;
  id  |   host    | port |     type     |  state  | parallelism | is_streaming | is_serving | is_unschedulable
------+-----------+------+--------------+---------+-------------+--------------+------------+------------------
 3001 | 127.0.0.1 | 5688 | COMPUTE_NODE | RUNNING |           4 | t            | t          | f
(1 row)

dev=> show clusters;
      Addr      |  State  |     Parallel Units     | Is Streaming | Is Serving | Is Unschedulable
----------------+---------+------------------------+--------------+------------+------------------
 127.0.0.1:5688 | RUNNING | 3000, 3001, 3002, 3003 | true         | true       | false
(1 row)

dev=> select * from rw_parallel_units ;
  id  | worker_id
------+-----------
 3000 |      3001
 3001 |      3001
 3002 |      3001
 3003 |      3001
(4 rows)

@codecov
Copy link

codecov bot commented Jun 29, 2023 •
edited
Loading

Codecov Report

Merging #10656 (297acee) into main (0ede620) will decrease coverage by 0.03%.
The diff coverage is 18.78%.

@@            Coverage Diff             @@
##             main   #10656      +/-   ##
==========================================
- Coverage   70.22%   70.19%   -0.03%     
==========================================
  Files        1280     1282       +2     
  Lines      220315   220452     +137     
==========================================
+ Hits       154718   154753      +35     
- Misses      65597    65699     +102
Flag Coverage Δ
rust 70.19% <18.78%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ntend/src/catalog/system_catalog/rw_catalog/mod.rs 0.00% <0.00%> (ø)
src/frontend/src/handler/show.rs 27.31% <0.00%> (-4.41%) ⬇️
src/meta/src/manager/cluster.rs 70.24% <ø> (ø)
src/sqlparser/src/ast/mod.rs 85.03% <0.00%> (-0.07%) ⬇️
src/sqlparser/src/keywords.rs 100.00% <ø> (ø)
src/sqlparser/src/parser.rs 86.81% <0.00%> (-0.03%) ⬇️
src/frontend/src/catalog/system_catalog/mod.rs 45.20% <100.00%> (+0.76%) ⬆️
.../catalog/system_catalog/rw_catalog/rw_functions.rs 100.00% <100.00%> (ø)
...rc/catalog/system_catalog/rw_catalog/rw_sources.rs 100.00% <100.00%> (ø)

... and 5 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

st1page
st1page reviewed Jun 30, 2023
View reviewed changes
@yezizp2012 yezizp2012 requested review from BugenZhao and st1page June 30, 2023 07:17
st1page
st1page reviewed Jun 30, 2023
View reviewed changes
Copy link
Contributor

@st1page st1page left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, up to you @BugenZhao

BugenZhao
BugenZhao approved these changes Jul 3, 2023
View reviewed changes
Copy link
Member

@BugenZhao BugenZhao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@yezizp2012 yezizp2012 added this pull request to the merge queue Jul 3, 2023
Merged via the queue into main with commit da25e2d Jul 3, 2023
@yezizp2012 yezizp2012 deleted the zp/rw-sys-cluster-table branch July 3, 2023 04:10
This was referenced Jul 11, 2023
Closed
Closed
@emile-00 emile-00 added the 📖✓ Covered or will be covered in the user docs. label Jul 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@BugenZhao BugenZhao BugenZhao approved these changes

@TennyZhuang TennyZhuang Awaiting requested review from TennyZhuang

@xiangjinwu xiangjinwu Awaiting requested review from xiangjinwu

+1 more reviewer

@st1page st1page st1page left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

type/feature Type: New feature. user-facing-changes Contains changes that are visible to users 📖✓ Covered or will be covered in the user docs.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@yezizp2012 @BugenZhao @st1page @emile-00