Releases · risingwavelabs/risingwave

For installation and running instructions, see Get started.

Main changes

SQL features

SQL command:
- Supports the SHOW CLUSTERS command. #10656
- Supports the GROUPING SETS clause. #10807
SQL function:
- Adds the current_setting() function to get the current value of a configuration parameter. #10051
- Adds new array functions: array_position(), array_replace(), array_ndims(), array_lower(), array_upper(), array_length(), and array_dims(). #10166, #10197
- Adds new aggregate functions: percentile_cont(), percentile_disc(), and mode(). #10252
- Adds new system functions: user(), current_user(), and current_role(). #10366
- Adds new string functions: left() and right(). #10765
- Adds new bytea functions: octet_length() and bit_length(). #10462
- array_length() and cardinality() return integer instead of bigint. #10267
- Supports the row_number window function that doesn't match the TopN pattern. #10869
User-defined function:
- Adds support for defining UDFs in Java. #10095
- Adds support for more Java UDF and Python UDF data types. #10399
- The language parameter is no longer required in CREATE FUNCTION. #10608
System catalog:
- Adds more columns to information_schema.columns: column_default, character_maximum_length, and udt_name. #10269
- Adds a new system catalog pg_proc. #10216
- Adds new RisingWave catalogs:
  - rw_table_fragments, rw_fragments, rw_actors #10712
  - rw_worker_nodes, rw_parallel_units #10656
  - rw_connections, rw_databases, rw_functions, rw_indexes, rw_materialized_views, rw_schemas, rw_sinks, rw_sources, rw_tables, rw_users, rw_views #10593
Supports GROUP BY output alias or index. #10305
Supports using scalar functions in the FROM clause. #10317
Supports tagging the created VPC endpoints when creating a PrivateLink connection. #10582

Connectors

Breaking change: When creating a source or table with a connector whose schema is auto-resolved from an external format file, the syntax for defining primary keys within column definitions is replaced with the table constraint syntax. #10195

CREATE TABLE debezium_non_compact (order_id int PRIMARY KEY) WITH (
connector = 'kafka',
kafka.topic = 'debezium_non_compact_avro_json',
kafka.brokers = 'message_queue:29092',
kafka.scan.startup.mode = 'earliest'
) ROW FORMAT DEBEZIUM_AVRO ROW SCHEMA LOCATION CONFLUENT SCHEMA REGISTRY 'http://message_queue:8081';

CREATE TABLE debezium_non_compact (PRIMARY KEY(order_id)) WITH ( ...

Breaking change: Modifies the syntax for specifying data and encoding formats for a source in CREATE SOURCE and CREATE TABLE commands. For v1.0.0, the old syntax is still accepted but will be deprecated in the next release. #10768

Old syntax - part 1:

ROW FORMAT data_format 
[ MESSAGE 'message' ]
[ ROW SCHEMA LOCATION ['location' | CONFLUENT SCHEMA REGISTRY 'schema_registry_url' ] ];

New syntax - part 1:

FORMAT data_format ENCODE data_encode (
    message = 'message',
    schema_location = 'location' | confluent_schema_registry = 'schema_registry_url'
);

Old syntax - part 2:

ROW FORMAT csv WITHOUT HEADER DELIMITED BY ',';

New syntax - part 2:

FORMAT PLAIN ENCODE CSV (
    without_header = 'true',
    delimiter = ','
);

Supports sinking data to Delta Lake. #10374, #10580
Supports sinking data to AWS Kinesis. #10437
Supports BYTES as a row format. #10592
Supports specifying schema for the PostgreSQL sink. #10576
Supports using the user-provided publication to create a PostgreSQL CDC table. #10804

Full Changelog: v0.19.0...v1.0.0

release v0.19.3

release v0.19.2

release v0.19.1

For installation and running instructions, see Get started.

Main changes

Installation

Now, you can easily install RisingWave on your local machine with Homebrew by running brew install risingwave.

Administration

Adds the pg_indexes and dattablespace system catalogs. #9844, #9822
Now, the SHOW PARAMETERS command will display the mutability of each system parameter. #9526

SQL features

Experimental features: Adds support for 256-bit integer. #9146, #9184, #9186, #9191, #9217
Indexes can be created on expressions. #9142
Adds support for expressions in aggregate function arguments. #9955
Adds support for VALUES clause. #8751
Adds support for generated columns, which are special columns computed from other columns in a table or source. #8700, #9580
Adds support for using expressions in the ORDER BY and PARTITION BY clauses. #9827
New SQL commands
- CREATE CONNECTION and SHOW CONNECTIONS: Creates an AWS PrivateLink connection and show all available connections. #8907
- DROP CONNECTION: Drops a connection. #9128
- SHOW FUNCTIONS: Shows existing user-defined functions. #9398
- DROP FUNCTIONS: Drops a user-defined function. #9561
- SHOW CREATE SOURCE and SHOW CREATE SINK: Shows the SQL statement used to create a source or sink. #9083
- SHOW INDEXES: Shows all indexes on a particular table. #9835
SQL functions
- Adds support for trigonometric functions. #8838, #8918, #9064, #9203, #9259
- Adds support for degrees and radians functions. #9108
- Adds support for the lag() and lead() window functions and the OVER() and EMIT ON WINDOW CLOSE clause. #9597, #9622, #9701
- Adds support for new aggregate functions, including bool_and, bool_or, jsonb_agg, and jsonb_object_agg. #9452
- Adds support for max(), min(), and count() for timestamptz data. #9165
- Adds support for microseconds and milliseconds for to_char() and to_timestamp(). #9257
- Adds support for multibyte Unicode in overlay() and ascii() functions. #9321
- Adds support for the string_to_array() function. #9289
- Adds support for array_positions(). #9152
- Adds support for cardinality(). #8867
- Adds support for array_remove(). #9116
- Adds support for trim_array(). #9265
- Adds support for array range access. #9362
- Adds support for JSONB in UDF. #9103
- Adds support for btrim() and updates trim() to PostgreSQL standard syntax. #8985
- Adds support for date_part(). #8830
- Expands extract() with more fields. #8830
- Adds support for proctime(), which returns the system time with time zone when a record is processed. #9088
- Adds support for translate(), @(), and ceiling(). #8998
- Adds support for encode() and decode(). #9351
- Adds support for the intersect operator. #9573
- Adds support for the default escape \ in a like expression. #9624
- Adds support for the IS [NOT] UNKNOWN comparison predicate. #9965
- Adds support for the starts_with() string function and ^@. #9967
- Adds support for unary trunc, ln, log10 (log), exp, cbrt (||/) mathematical functions. #9991

Connectors

Adds support for ingesting CDC data from TiDB and sinking data to TiDB with the JDBC connector. #8708
Adds support for ingesting CDC data from Citus. #8988
Adds support for loading Pulsar secret key file from AWS S3. #8428, #8222
Adds support for using an established AWS PrivateLink connection in a CREATE SOURCE, CREATE TABLE, or CREATE SINK statement for a Kafka source/sink. #9119, #9128, #9728, #9263
Deprecates the use_transaction field in the Kafka sink connector. #9207
Adds support for zstd compression type for Kafka connector. #9297
Deprecates the upsert property in the Kafka connector as it can be inferred from the row format. #9457
Adds a new field properties.sync.call.timeout in the WITH clause of the Kafka source connector to control the timeout. #9005
Adds support for a new row format DEBEZIUM_MONGO_JSON in the Kafka source connector. #9250
Adds CSV format support for the Kafka source connector. #9875

Cluster configuration changes

--data_directoryand --state_storemust be specified on CLI of the meta node, or the cluster will fail to start. #9170
Clusters will refuse to start if the specified object store URL identified by state_store and data_directory is occupied by another instance. Do not share the object store URL between multiple clusters. #9642

Full Changelog: v0.18.0...v0.19.0

For installation and running instructions, see Get started.

Main changes

Starting from this version, we’ll respect semantic versioning conventions by using the middle number (y , instead of z, in x.y.z) to indicate minor versions. That is why this is v0.18.0, not v0.1.18.

Administration and troubleshooting

Improves error messages by including the location of the statement in question. #8646
Initial values of immutable system parameters can be specified via the meta-node command line. The initial values provided in the configuration file will be ignored. #8366

SQL features

Adds initial support for user-defined functions. #8597 #8644 #8255 #7943
Adds support for JSONB data type. #8256 #8181
Adds support for NULLS { FIRST | LAST } in ORDER BY clauses. #8485
New commands:
- ALTER SOURCE RENAME TO #8778
- SET TIME ZONE #8572
- ALTER RELATION RENAME #7745
- ALTER TABLE ADD/DROP COLUMN for regular tables (without connector settings). #8394
New functions:
- array_length : Returns the length of an array. #8636
- String functions implemented with the help of chatGPT. #8767 #8839
  - chr(integer) -> varchar
  - starts_with(varchar, varchar) -> boolean
  - initcap(varchar) -> varchar
  - lpad(varchar, integer) -> varchar
  - lpad(varchar, integer, varchar) -> varchar
  - rpad(varchar, integer) -> varchar
  - rpad(varchar, integer, varchar) -> varchar
  - reverse(varchar) -> varchar
  - strpos(varchar, varchar) -> integer
  - to_ascii(varchar) -> varchar
  - to_hex(integer) -> varchar
  - to_hex(bigint) -> varchar)
- Improves the data type values of columns returned by DESCRIBE . #8819
- UPDATE commands cannot update primary key columns. #8569
- Adds support for microsecond precision for intervals. #8501
- Adds an optional parameter offset to tumble() and hop() functions. #8490
- Data records that has null time values will be ignored by time window functions. #8146
- Improves the behaviors of the exp operator when the operand is too large or small. #8309
- Supports process time temporal join, which enables the joining of an append-only stream (such as Kafka) with a temporal table (e.g. a materialized view backed by MySQL CDC). This feature ensures that any updates made to the temporal table will not affect previous results obtained from the temporal join. Supports FOR SYSTEM_TIME AS OF NOW() syntax to express process time temporal join. #8480

Connectors

Adds a new field basetime to the load generator connector for generating timestamp data. The load generator will take this field as now and generates data accordingly. #8619
Empty cells in CSV are now parsed as null. #8709
Adds the Iceberg connector. #8508
Adds support for the upsert type to the Kafka sink connector. #8168
Removes the message name parameter for Avro data. #8124
Adds support for AWS PrivateLink for Kafka source connector. #8247

Full Changelog: v0.1.17...v0.18.0

For installation and running instructions, see Get started.

Main changes

Administration

Adds a system catalog view rw_catalog.rw_ddl_progress, with which users can view the progress of a CREATE INDEX, CREATE SINK, or CREATE MATERIALIZED VIEW statement. #7914
Adds the pg_conversion and pg_enum system catalogs. #7964, #7706

SQL features

Adds the exp() function. #7971
Adds the pow() function. #7789
Adds support for displaying primary keys in EXPLAIN statements. #7590
Adds support for descending order in CREATE INDEX statements. #7822
Adds SHOW PARAMETERS and ALTER SYSTEM commands to display and update system parameters. #7882, #7913

Connectors

Adds a new parameter match_pattern to the S3 connector. With the new parameter, users can specify the pattern to filter files that they want to ingest from S3 buckets. For documentation updates, see Ingest data from S3 buckets. #7565
Adds the PostgreSQL CDC connector. Users can use this connector to ingest data and CDC events from PostgreSQL directly. For documentation updates, see Ingest data from PostgreSQL CDC. [#6869](#6869, #7133
Adds the MySQL CDC connector. Users can use this connector to ingest data and CDC events from MySQL directly. For documentation updates, see Ingest data from MySQL CDC. #6689, #6345, #6481, #7133
Adds the JDBC sink connector, with which users can sink data to MySQL, PostgreSQL, or other databases that are compliant with JDBC. #6493
Add new parameters to the Kafka sink connector.
- force_append_only : Specifies whether to force a sink to be append-only. #7922
- use_transaction : Specifies whether to enable Kafka transactions or not. #7500
- SSL/SASL parameters: Specifies SSL encryption and SASL authentication settings. #7540

Full Changelog: v0.1.16...v0.1.17

For installation and running instructions, see Get started.

Main changes

Administration

Adds support for aborting a query in local mode with Ctrl + C. #7444

SQL features

Adds support for the to_timestamp function. #7060
Adds support for the RETURNING clause in DML statements. #7094
Breaking change: Deprecates CREATE MATERIALIZED SOURCE . To create a materialized source, create a table and include the newly added connector settings. #7281, #7110
Adds support for the c and i flags in regex_match() and regex_matches() functions. #7135
Adds support for SHOW CREATE TABLE . You can use this statement to show the definition of a table. #7152
Adds support for the pg_stat_activity system catalog and several system functions. #7274
Adds the _rw_kafka_timestamp parameter to show the timestamps of Kafka messages. Users can now specify the scope of Kafka messages by timestamps. #7275, #7150
Adds support for displaying PostgreSQL and RisingWave versions in version(). #7314
Adds support for displaying internal tables using the SHOW INTERNAL TABLES statement. #7348
Adds support for SET VISIBILITY_MODE You can use this session variable to configure whether only checkpoint data is readable for batch query. #5850
Adds support for SET STREAMING_PARALLELISM . You can use this session variable to configure parallelism for streaming queries. #7370

Connectors

Adds support for generating array and struct data using the datagen connector. #7099
Adds the S3 source connector, with which users can ingest data in CSV format from S3 locations. For data ingestion from files, CSV is the only supported format and the files must be placed on S3. #6846

Full Changelog: v0.1.15...v0.1.16

For installation and running instructions, see Get started.

Main changes

Installation and deployment

Parallelism and available memory of compute nodes are now command-line arguments and removed from the configuration file. #6767
The default barrier interval is set to 1 second. #6553
Adds support for meta store backup and recovery. #6737

SQL features

Adds support for SHOW CREATE MATERIALIZED VIEW and SHOW CREATE VIEW to show how materialized and non-materialized views are defined. #6921
Adds support for CREATE TABLE IF NOT EXISTS. #6643
A sink can be created from a SELECT query. #6648
Adds support for struct casting and comparison. #6552
Adds pg_catalog views and system functions. #6982
Adds support for CREATE TABLE AS. #6798
Ads the initial support for batch query on Kafka source. #6474
Adds support for SET QUERY_EPOCH to query historical data based on meta backup. #6840

Connectors

Improves the handling of schema errors for Avro and Protobuf data. #6821
Adds two options to the datagen connector to make it possible to generate increasing timestamp values. #6591

Observability

Adds metrics for the backup manager in Grafana. #6898
RisingWave Dashboard can now fetch data from Prometheus and visualize it in charts. #6602

Full Changelog: v0.1.14...v0.1.15

For installation and running instructions, see Get started.

Main changes

SQL features

PRIMARY KEY constraint checks can be performed on materialized sources and tables but not on non-materialized sources. For tables or materialized sources that enabled PRIMARY KEY constraints, if you insert data to an existing key, the new data will overwrite the old data. #6320 #6435
Adds support for timestamp with time zone data type. You can use this data type in time window functions, and convert between it and timestamp (without time zone). #5855 #5910 #5968
Adds support for UNION and UNION ALL operators. #6363 #6397
Implements the rank() function to support a different mode of Top-N queries. #6383
Adds support for logical views (CREATE VIEW). #6023
Adds the data_trunc() function. #6365
Adds the system catalog schema. #6227
Displays error messages when users enter conflicting or redundant command options. #5933

Connectors

Adds support for the Maxwell Change Data Capture (CDC) format. #6057
Protobuf schema files can be loaded from Web locations in s3://, http://, or https:// formats. #6114 #5964
Adds support for Confluent Schema Registry for Kafka data in Avro and Protobuf formats. #6289
Adds two options to the Kinesis connector. Users can specify the startup mode and optionally the sequence number to start with. #6317

Full Changelog: v0.1.13...v0.1.14

Releases: risingwavelabs/risingwave

v1.0.0

Main changes

SQL features

Connectors

Uh oh!

v0.19.3

Uh oh!

v0.19.2

Uh oh!

v0.19.1

Uh oh!

v0.19.0

Main changes

Installation

Administration

SQL features

Connectors

Cluster configuration changes

Uh oh!

v0.18.0

Main changes

Administration and troubleshooting

SQL features

Connectors

Uh oh!

v0.1.17

Main changes

Administration

SQL features

Connectors

Uh oh!

v0.1.16

Main changes

Administration

SQL features

Connectors

Uh oh!

v0.1.15

Main changes

Installation and deployment

SQL features

Connectors

Observability

Uh oh!

v0.1.14

Main changes

SQL features

Connectors

Uh oh!