Breaking Changes

exonum-api

• ApiManagerConfig was made non-exhaustive. (#1834)

New Features

exonum-node

• Exonum nodes now gracefully terminate on receiving SIGINT, SIGTERM
  and SIGQUIT signals (on Unix platforms), or a ctrl + c break (on Windows).
  These signal handlers may be switched off by using NodeBuilder::disable_signals().
  (#1834)

Bug Fixes

exonum-cli

• Fixed bug in run-dev command. Previously, using the existing configuration
  files in blockchain-path led to an error. (#1827)

Breaking changes

exonum

• create_patch and commit methods in BlockchainMut have been generalized
  to support block skipping (see New Features section for more details). (#1820)

exonum-cli

• run-dev command has been reworked. It now does not clear database files
  after the launch. artifacts-dir parameter has been renamed to
  blockchain-path. Configuration files are now stored inside config
  subdirectory. (#1822)

exonum-supervisor

• MigrationRequest was made non-exhaustive. (#1823)

• POST endpoints now expect JSON-encoded input rather than
  hex-encoded Protobuf. (#1823)

• supervisor_name method was removed. Use Supervisor::NAME instead. (#1823)

New Features

exonum-cli

• Several constants in the command module became public. (#1821)

exonum-node

• Exonum nodes can now customize how they create block proposals. This can be
  used to whitelist / blacklist transaction authors or services, prioritize
  transactions by advanced criteria, implement complex rate limiting, etc.
  The functionality is available via proposer module. (#1820)

• Exonum nodes can now skip block generation at a certain epoch of the consensus
  algorithm. This can be used to keep a "heartbeat" when the network load is low
  without bloating the storage used by the nodes. (#1820)

exonum-rust-runtime

• ServiceApiScope::pb_endpoint_mut allows to accept Protobuf-encoded messages
  with the request content type set to application/octet-stream in addition
  to JSON-encoded messages. (#1829)

exonum-testkit

• Testkit can send Protobuf-encoded payloads to POST endpoints. (#1831)

Bug Fixes

exonum-api

• Introduced a workaround for the HTTP restart hanging up on Windows. (#1828)

exonum-node

• Fixed a bug when a node created a propose with incorrect transactions.
  This could lead to consensus failure or weird error messages in the node log.
  (#1820)

exonum-rust-runtime

• Fixed updating HTTP endpoints if the Rust runtime does not contain
  active services during node start. (#1831)

exonum-supervisor

• DeployRequest and MigrationRequest now have cryptographic seeds
  to retry the same request multiple times. (#1823)

Breaking changes

General

• Error handling is now performed with the anyhow crate instead of failure.
  (#1805)

• APIs which previously used futures from the futures 0.1 crate are now
  made async, or are using Futures from the standard library. (#1796, #1804)
  The main affected APIs are as follows:

  • Endpoint definitions in the exonum-api crate and their counterparts
    in exonum-rust-runtime
  • Transaction sending with ApiSender from the exonum crate
  • Node start-up with Node::run (exonum-node crate) and NodeBuilder::run
    (exonum-cli crate), and shutdown with ShutdownHandle (exonum-node crate)
  • HTTP API testing with TestKitApi (exonum-testkit crate)

exonum

• Testkit now does not include incorrect transactions into blocks or memory pool,
  similar to real Exonum nodes. (#1785)

• Runtime::deploy_artifact no longer returns Box<dyn Future<...>>. Instead a
  special communication channel is used to send deployment status from the
  runtime to the dispatcher. (#1788)

• Schema::call_errors was removed in favor of more comprehensive
  call_records method. (#1792)

• Blockchain::create_patch and Blockchain::commit signatures were changed
  due to unsoundness of the previous implementation; see "Bug Fixes" section
  for more details. (#1809)

• Replaced CoreError::ServiceNotStopped with the more general InvalidServiceTransition
  error. (#1806)

exonum-api

• Data types were made non-exhaustive where appropriate. (#1799)

exonum-cli

• NodeBuilder was refactored to use a more intuitive set of interfaces
  for adding built-in artifacts and services to the blockchain. (#1800)
• Submodules of the command module were made private; the relevant data types
  are now exported from the command module directly. Similarly,
  io module was made private. (#1799)

exonum-explorer

• Data types were made non-exhaustive where appropriate. (#1799)

exonum-proto

• impl_binary_value_for_pb_message macro was removed. Use the BinaryValue
  derive macro from the exonum-derive crate instead. (#1805)

exonum-rust-runtime

• Data types were made non-exhaustive where appropriate. (#1799)

exonum-supervisor

• Data types were made non-exhaustive where appropriate. (#1799)

exonum-system-api

• Public api module has been removed. List of endpoints from private api has
  been changed. (#1790) Current list of the endpoints:

  • v1/info - obtains information about the node;
  • v1/stats - obtains statistics of the node;
  • v1/peers - adds a peer to the Exonum node;
  • v1/consensus_status - enables or disables consensus on the node;
  • v1/shutdown - shuts down the node.

• API data types were made non-exhaustive where appropriate. (#1799)

exonum-testkit

• TestKitBuilder was refactored to use a more intuitive set of interfaces
  for adding built-in artifacts and services to the blockchain. (#1800)

• Data types were made non-exhaustive where appropriate. (#1799)

New Features

exonum

• Exonum now supports service freezing. A frozen service has fixed state (i.e.,
  does not process transactions and service hooks), but, unlike stopped services,
  the service HTTP API remains active. (#1780)

• Core and the explorer service now support retrieving call status with
  a cryptographic proof of authenticity. (#1792)

• Exonum now supports unloading of unused service artifacts. This operation
  may be used to free resources associated with artifacts in the runtime
  hosting them. (#1794)

exonum-supervisor

• Supervisor service supports service freezing. (#1781)

• supervisor/services endpoint has been added which obtains information
  about deployed artifacts and available services. (#1790)

• Supervisor service supports artifact unloading. (#1798)

exonum-rust-runtime

• Rust services support freezing. (#1780)

• HTTP API of Rust services is now switched on during data migrations. (#1780)

exonum-testkit

• Testkit server now returns info on emulated nodes. (#1799)

Internal Improvements

exonum

• Core now provides more thorough / context-dependent error descriptions
  related to service lifecycle. (#1806)

exonum-merkledb

• Index clearing now uses range deletions for RocksDB, providing
  a significant performance boost for this operation. (#1791)

Bug Fixes

exonum

• Fixed bug related to nodes forgetting transactions after executing
  a block with them. Previously, nodes forgot all transactions
  in the executed blocks; such transactions were removed from
  the ephemeral transaction cache, but were not flushed to the DB
  or anywhere else. This could lead to consensus hang-up. (#1809)

exonum-node

• Fixed potential node hang-up if the node received a proposal and
  a supermajority of transactions approving it before all transactions
  in the proposal are known. (#1781)

• Switched off broadcasting of transactions which the node considers
  incorrect. (#1781)

• Fixed incorrect invalidation of block proposals. (#1782)

• Provided clear coherence period for the transaction pool
  by introducing the mempool.flush_config_strategy configuration parameter.
  Previously, transactions were flushed to the persistent pool
  only on block commit. This led to the unexpected behavior of some APIs,
  such as the transaction getter endpoint in the explorer service. (#1809)

• Fixed race condition when two nodes try to establish outgoing connections
  to each other at the same time. (#1804)

exonum-testkit

• Testkit now does not include incorrect transactions into blocks or memory pool,
  similar to real Exonum nodes. (#1785)

• Added a method to get reference to the underlying API client for the TestKitApi.
  (#1811)

This release marks the significant milestone in Exonum evolution – the minimum viable lifecycle for Exonum services. Besides dynamic deployment of services implemented in 0.13.0-rc.2 release, Exonum now supports stopping and upgrading service instances (including the possibility of data migrations). Like other lifecycle events, stopping and upgrades are implemented by the core and controlled by the supervisor service, providing flexibility and upgradeability of the control logic.

As a part of service upgrade workflow, Exonum now supports background (aka asynchronous) data migrations. This allows to migrate data from one schema to the other without stopping a node or the blockchain network; the whole process is performed in the background while the node continues processing transactions and other requests. Relevant migration support was added to the storage backend (MerkleDB) and can be used independently.

Other significant changes:

• The crate structure was significantly refactored. The node logic, Rust runtime, HTTP API wrapper, explorer endpoints and system endpoints, all of which were previously parts of the main crate, were moved to separate crates. This provides greater flexibility for core and service developers and allows to evolve downstream crates independently of the core.
• MerkleDB now performs fully automated state aggregation, requiring zero efforts from the core and services. The aggregation logic in core and runtimes was thus removed.
• MerkleDB now supports generic database accesses and shared iterator types for all indexes, allowing to simplify the support code for bindings.
• Service interfaces were reworked in the Rust runtime; now, they are fully represented by traits. These interface traits are automatically implemented for an assortment of stub types, such as keypairs; more stub types can be defined by a user. See the stubs module in the Rust runtime crate for a literary code explainer.
• Preparations have been made to make potential changes to the core interfaces non-breaking. Relevant structures and enumerations were made non-exhaustive, and some unstable interfaces were marked as such in the documentation (or hidden from the docs).

See more detailed list of changes in CHANGELOG.md.

Stability Notice

No changes to the core functionality are expected between this release candidate and the 1.0.0 release. At the same time, there may be breaking changes to the APIs documented as unstable (for example, data migration). Another kind of potential breaking changes is related to updating core dependencies. Right now, Exonum core targets compatibility with Rust 1.36 due to the problems of compiling the Java runtime with the more recent versions. If this restriction is lifted, some core dependencies will be updated, which could result in breaking changes (at least technically).

Breaking changes

exonum

• Most important: new Dynamic Services feature was introduced. For details see
  the Dynamic Services section of the changelog.

• Used rust version is updated to 1.38.0. (#1481)

• create_checkpoint method has been implemented for the RocksDB struct.
  This method uses RocksDB checkpoints functionality under the hood.

• NotFound error message for explorer/v1/block endpoint now includes
  the actual blockchain height. (#1498)

• system/v1/rebroadcast endpoint has been removed. (#1445)

• Added a possibility to specify compression algorithm for the database. (#1447)

• Updated hex dependency with changes in the methods signatures of the ToHex
  trait. (#1468)

• Validator keys are now derived from single master key. Master key is
  stored in encrypted file. (#1459)

• Command line parameters --service-key-pass and --consensus-key-pass was
  removed in favor of --master-key-pass parameter. For example now you can
  run the node with the command below. (#1459)

  cargo run -- run -d 0/db/ -c 0/node.toml --master-key-pass pass:123

  • StoppedTestKit::resume accepts list of runtimes instead of a list of services.

  • Removed obsolete TestKit::blockchain_mut method and TestKit::blockchain
    now returns value instead of reference.

• Dot symbol is not allowed in service names anymore. (#1558)

• Services can now use BlockchainData and SnapshotExt types to access data
  from the blockchain in a more structured manner. (#1523)

• GenesisConfig is extracted into separate entity. BlockchainBuilder, Node
  and Testkit explicitly accepts it during creation. (#1541)

• Added DefaultInstance trait for declaration of builtin services. (#1541)

exonum-merkledb

• Nested proofs for ProofListIndex are replaced with a flat
  (non-recursive) format. (#1450)

• Differentiated (read-only / read-write) access to the database
  was introduced. (#1523)

• It is now possible to have readonly access to indexes given a Fork
  via a ReadonlyFork wrapper. Readonly access works like RefCell::borrow
  (vs RefCell::borrow_mut for Fork); it is possible to create an
  unlimited number of indexes with readonly access based on the same fork.
  (#1523)

• Service schemas can now use a declarative layout, in which every field
  corresponds to a separate index or a group of indexes. It is possible
  to derive a constructor for such schemas via FromAccess derive macro.
  (#1523, #1562)

• New index name restrictions has been added. (#1558)

  • Dot symbol is not allowed anymore in indexes with prefixed access.

  • Index names starting from __ and not containing a dot . are reserved and
    used only for system indexes.

exonum-proto

• Introduced a new crate exonum-proto. Trait ProtobufConvert is moved
  to this crate. (#1496)

exonum-protobuf-convert

• Introduced a new crate exonum-protobuf-convert. Derive macro
  ProtobufConvert is moved to this crate. (#1501)

• Derive macro ProtobufConvert now does not derive the BinaryValue and
  ObjectHash traits. There are separate derive macros for them in
  the exonum-derive crate. (#1501)

exonum-build

• Method protobuf_generate is now private, use exonum_build::ProtobufGenerator
  instead (#1496).

• Method ProtobufGenerator::frequently_used has been removed (#1581).

exonum-crypto

• Methods read_keys_from_file and generate_keys are moved to new keys
  module in the exonum. (#1459)

• Protobuf serialization for crypto types is now implemented in the exonum-crypto
  crate. (#1496)

Dynamic Services Feature

Overview

In exonum 0.13.0-rc.2, a new service workflow is introduced, named
"Dynamic Services".

Key points of this feature are the following:

• exonum now supports different environments of code execution (runtimes).
  Only native rust runtime is enabled by default, but support of
  different programming languages can be added quite easily.

  For details see the Runtime trait docs and the
  sample_runtime example.

• Services are not statically tied to the compiled binary anymore. There is
  support of adding new service types (aka artifacts) dynamically and starting new
  instances of services.

  For details see runtime module docs.

• Services now can have initialization parameters, provided within service start
  procedure.

• Services now support configuration changes via Configure interface.

• configuration service was replaced with the supervisor service, which is
  capable of not only changing configuration, but of deploying and starting
  services as well. For details see supervisor service.

Migration Guide

There are a lot of backward-incompatible changes introduced within 0.13.0-rc.2 release candidate.
So to make the changes apparent, compare the Cryptocurrency example service versions
for 0.12.1 and 0.13.0 releases.

Key points:

• Merkledb schema is now declarative and can contain indices as fields.

• Access to the database is now isolated for services.
  A service cannot get the write access to another service or the blockchain schema.

• Transactions do not have the execute method anymore. Instead, a service defines
  and implements an interface trait which contains all the business logic.

• Services do not launch at the node start by default. For launching a
  service, use an exonum-launcher tool.

Important PRs for Dynamic Services

Full History of the Dynamic Services Implementation

exonum

Bug Fixes

• A message length checking has been fixed (#1463)

Internal Improvements

exonum

• Update dependencies.

exonum-crypto

• Update dependencies.

Breaking changes

exonum

• Signatures of methods Service::initialize and Service::before_commit has been
  changed. Now they take immutable reference to Fork instead of mutable. (#1293)

• Trait BinaryForm has been replaced by BinaryValue. (#1298)

  To implement BinaryValue for types that implements Protobuf::Message use
  impl_binary_value_for_pb_message macros.

• Module storage has been replaced by exonum-merkledb crate. See related section
  in changelog for details. (#1293)

• Bootstrapping workflow has been simplified (#1292)

  generate-config subcommand now uses single OUTPUT_DIR instead of set of options.
  So to generate node config you should write something like example bellow.

  cargo run --bin exonum-timestamping -- \
    generate-template /tmp/exonum/template.toml --validators-count 4
  
  cargo run --bin exonum-timestamping -- \
    generate-config /tmp/exonum/template.toml /tmp/exonum/cfg/0 \
      --peer-address 0.0.0.0:8000
  cargo run --bin exonum-timestamping -- \
    generate-config /tmp/exonum/template.toml /tmp/exonum/cfg/1 \
      --peer-address 0.0.0.0:8001
  cargo run --bin exonum-timestamping -- \
    generate-config /tmp/exonum/template.toml /tmp/exonum/cfg/2 \
      --peer-address 0.0.0.0:8002
  cargo run --bin exonum-timestamping -- \
    generate-config /tmp/exonum/template.toml /tmp/exonum/cfg/3 \
      --peer-address 0.0.0.0:8003
  
  cargo run --bin exonum-timestamping -- \
    finalize /tmp/exonum/nodes/0/sec.toml /tmp/exonum/nodes/0/node.toml \
      --public-configs /tmp/exonum/cfg/{0,1,2,3}/pub.toml
  
  cargo run --bin exonum-timestamping -- \
    run -d /tmp/exonum/db/0 -c /tmp/exonum/nodes/0/node.toml

• explorer/v1/blocks endpoint with add_blocks_time param switched on now returns
  median precommit times in the time field within each returned block,
  rather than in a separate array. (#1278)

• system/v1/mempool endpoint has been renamed into system/v1/stats.
  An additional field in the response of the endpoint was added. The field
  corresponds to the total number of transactions in the blockchain. (#1289)

exonum-merkledb

• Added restrictions to index names. Allowable characters in index name: ASCII
  characters, digits, underscores and dashes. (#1388)

• Added Debug implementation for Database, Snapshot, Iterator dynamic
  traits (#1363)

• Changed storage layout (#1293)

  • Changed indexes metadata layout in the database.

  • Introduced a generic IndexState structure that can be used to store global
    index properties like total number of items.

• Changed ProofMapIndex hashing rules for branch nodes and root node.
  Branch nodes is hashing now with 0x04 prefix, root node with 0x03 (#1293).

• Renamed method merkle_root of ProofMapIndex and ProofListIndex to
  object_hash (#1293).

• Several mutable indexes now can be create from immutable reference to Fork (#1293)

• Relaxed trait bounds for the ProofMapIndex keys (#1293)

  Now keys should just implement BinaryKey trait instead of the
  ProofMapKey, which will be ordered according to their binary
  representation, as in the MapIndex.

• Changed ProofListIndex hashing rules for leaf nodes and branch nodes according
  to the certificate transparency
  specification. Leaf nodes contain hashes with 0x00 prefix, branch nodes - with
  0x01. (#1293)

• StorageValue and StorageKey have been renamed to the BinaryValue
  and BinaryKey. (#1293)

  • Added to_bytes method to the BinaryValue trait which doesn't consume
    original value instead of the into_bytes.
  • BinaryKey::write now returns total number of written bytes.
  • CryptoHash has been replaced by the ObjectHash.

• Changed the hash algorithm of the intermediate nodes in ProofMapIndex. (#1293)

  ProofPath now uses compact binary representation in the BranchNode
  hash calculation.

  Binary representation is |bits_len|bytes|, where:

  • bits_len - total length of the given ProofPath in bits compressed
    by the leb128 algorithm
  • bytes - non-null bytes of the given ProofPath, i.e. the first
    (bits_len + 7) / 8 bytes.

New features

exonum

• New endpoint: v1/transactions/subscribe, which subscribe to new transaction events.
  This endpoint accept optional parameters: service_id and message_id
  (message_id as in derive macro TransactionSet). (#1335)

• New endpoint: v1/ws, which open websocket connection and allow to set multiple
  subscription (for blocks and transaction, filtered by service and transaction id)
  and send transactions (in hex, like in explorer) to blockchain
  (examples can be found in related pull request). (#1335)

Bug Fixes

exonum-testkit

• Fixed TestKit::add_tx() method, which previously did not persist
  transactions. (#1278)

Internal improvements

exonum

• explorer/v1/blocks endpoint supports add_precommits param, which supplies
  each returned block with the precommits field. (#1278)

• explorer/v1/blocks endpoint allows to specify the lower bound on the returned
  block height with the earliest query param. (#1278)

• Added ProtobufConvert implementation for byte array with fixed sizes (#1279)

• Added service_name getter to the TransactionContext. (#1274)

• Allowed to use symbol - in index names. (#1277)

• rocksdb crate is now used instead of exonum_rocksdb. (#1286)

• Added a new endpoint system/v1/services for displaying information
  about available services. (#1288)

• Endpoints explorer/v1/block and explorer/v1/transactions were extended
  with adding additional fields service_id and time. (#1386)

• Added tx_cache field to State to cache incoming transactions before
  adding them to persistent pool. (#1398)

• Added new request message PoolTransactionsRequest to obtain pool transactions
  from another peers. (#1404)

exonum-merkledb

• Updated ProofMapIndex data layout. (#1293)

  Path to the root node in merkle patricia tree now has been stored in the index
  state.

• New API for getting and creating indexes. (#1293)

  • Now indexes can be accessed via immutable references from Snapshot and
    mutable/immutable references from Fork.

  • Introduced method fork::get_object to get or create object by address.

  • get_object_existed and get_object_existed_mut methods of Fork and Snapshot
    returns optional references to index.

• rocksdb crate is now used instead of exonum_rocksdb. (#1286)

• Added From<Patch> trait implementation to Fork. (#1403)

exonum-testkit

• Implemented "stopping" and "resuming" a TestKit, allowing to emulate node
  restarts. (#1278)

Breaking Changes

exonum

• Node secret keys are now stored in separate files in a secure way.
  CLI for generating node configs and starting nodes has been extended
  in order to reflect these changes. (#1222, #1096, #1235)

• Changed a response for /healthcheck endpoint. (#1252)

• Changed a response code for the /block endpoint for the case when
  the requested block doesn't exist. (#1262)

• Removed a sub-command generate-testnet from CLI. (#1264)

exonum-crypto

• Renamed create_keys_file function to generate_keys_file
  in utils module. (#1222, #1096)

Internal improvements

• All Exonum crates have been updated to Rust 2018 edition. This means that
  it is required to use Rust 1.31 or newer for compilation. (#1230)

exonum

• Added allow-origin for localhost for public and private api in
  development mode by default. (#1234)

• Added ProtobufConvert implementation for Signature. (#1241)

Internal Improvements

exonum

• Unpin versions of dependencies.