Skip to content

tivnet/liboffkv

 
 

Repository files navigation

liboffkv

License Build Status

The library is designed to provide a uniform interface for three distributed KV storages: etcd, ZooKeeper and Consul.

The services have similar but different data models, so we outlined the common features.

In our implementation, keys form a ZK-like hierarchy. Each key has a version that is uint64 number greater than 0. Current version is returned with other data by the most of operations. All the operations supported are listed below.

Our library has only an asynchronous interface, but if you want a synchronous one, just call get() on the returned futures.

Method Parameters Description
create key: string
value: char[]
leased: bool (=false) -- makes the key to be deleted on client disconnect
Creates the key.
Throws an exception if the key already exists or
preceding entry does not exist.
set key: string
value: char[]
Assigns the value.
Creates the key if it doesn’t exist.
Throws an exception if preceding entry does not exist.
cas key: string
value: char[]
version: uint64 (=0) -- expected version of the key
Compare and set operation.
If the key does not exist and version equals 0 creates it.
Throws an exception if preceding entry does not exist.
If the key exists and its version equals to specified one updates the value. Otherwise does nothing.
get key: string
watch: bool (=false) -- start watching for change in value
Returns the value currently assigned to the key.
Throws an exception if the key does not exist.
If watch is true, returns a future that will be waiting til
the value is changed (see an example below).
exists key: string
watch: bool (=false) -- start watching for removal or creation of the key
Checks if the key exists.
If watch is true, returns a future that will be waiting for the key to be erased or created.
get_children key: string
watch: bool (=false)
Returns a list of the key's direct children.
Throws an exception if the key does not exist.
If watch is true, returns a future that will be waiting for any changes among the key's children.
erase key: string
version: uint64 (=0)
watch: bool (=false)
Erases the key and all its descendants if given version equals to the current key's version,
or does it unconditionally if version is 0.
Throws an exception if the key does not exist.
commit transaction: Transaction Commits transaction (see transactions API below).

Transactions

Transactions consist of 4 operations: create, set, erase, check. Their descriptions can be found below. At the moment set has different behavior in comparison to ordinary set: when used in transaction, set does not create key if it does not exist. Besides using any of the operations you cannot assign watch. Transaction body is separated into two blocks: firstly you should write all required checks and then the sequence of other operations (see an example below). As a result you get a list of new versions for all the keys involved in set operations.

Method Parameters Description
create key: string
value: char[]
leased: bool (=false)
Creates the key.
Rolls back if the key already exists or preceding entry does not exist.
set key: string
value: char[]
Set the value.
Rolls back if the key does not exist.
n.b behaviors of transaction set and ordinary set differ
erase key: string
version: uint64 (=0)
Erases the key and all its descendants
if given version equals to the current key's version,
or does it unconditionally if version is 0.
Rolls back if the key does not exist.
check key: string
version: uint64
Checks if given key has specified version
or only checks if it exists if version is 0

Usage

    #include <future>
    #include <iostream>

    #include "client.hpp"
    #include "error.hpp"
    #include "result.hpp"
    
    
    using namespace liboffkv;
    
    
    
    int main() 
    {
        // firstly specify protocol (zk | consul | etcd) and address to connect to the service
        // you can also specify a prefix all the keys will start with
        auto client = connect("consul://127.0.0.1:8500", "/prefix");
        
        // each method returns a future
        std::future<CreateResult> result = client->create("/key", "value");
        
        // sometimes it is returned with an exception (for more details see "liboffkv/client_interface.hpp"
        try {
            std::cout << "Key \"/prefix/key\" created successfully! "
                      << "Its initial version is " << result.get().version << std::endl;
        } catch (EntryExists&) {
            std::cout << "Error: key \"/prefix/key\" already exists!" << std::endl;
        }
	
	
	// WATCH EXAMPLE
	
	std::mutex lock;
    	lock.lock();
	
	// let's create a thread erasing key
    	std::thread([this, &lock]() mutable {
	    std::lock_guard<std::mutex> lock_guard(lock);
	    client->erase("/key").get();
        }).detach();
	
	// we call exists on newly create key "/prefix/key"
        auto result = client->exists("/key", true).get();
	
	// here we allow to launch erase job
        lock.unlock();

	// result is true
        assert(result);
	
	// here we wait untill "/prefix/key" is deleted
        result.watch.get();
	
	// and then result is false
        assert(!client->exists("/key").get());
        
	
	// TRANSACTION EXAMPLE
        
        // n.b. checks and other ops are separated from each other
        client->commit(
            {
                {
                    op::Check("/key", 42u),
                    op::Check("/foo"),
                },
                {
                    op::erase("/key"),
                    op::set("/foo", "new_value"),
                }
            }
        ).get();
    }

Supported platforms

The library is currently tested on

  • Ubuntu 18.04

    Full support.

  • MacOS

    Full support.

  • Windows 10

    Only Consul is supported.

Dependencies

  • C++ compiler

    Currently tested compilers are

    • VS 2019
    • g++ 7.4.0
    • clang

    VS 2017 is known to fail.

  • CMake

    We suggest using cmake bundled with vcpkg.

  • vcpkg

Developer workflow

  • Copy all ports from liboffkv/vcpkg/ports/ to [vcpkg root]/ports or create corresponding symbolic links.

    Note that the original vcpkg curl port is broken. Use fixed version from this repo.

  • Install dependencies

    # from vcpkg root
    vcpkg install ppconsul etcdcpp zkpp

    Installing all three packages is not required. See control flags at the next step.

  • Build tests

    # from liboffkv directory
    mkdir cmake-build-debug && cd $_
    cmake -DCMAKE_BUILD_TYPE=Debug \
          -DCMAKE_TOOLCHAIN_FILE="<replace with path to vcpkg.cmake>" \
          -DBUILD_TESTS=ON ..
    cmake --build .

    You can control the set of supported services with the following flags

    • -DENABLE_ZK=[ON|OFF]
    • -DENABLE_ETCD=[ON|OFF]
    • -DENABLE_CONSUL=[ON|OFF]

    Sometimes you may also need to specify VCPKG_TARGET_TRIPLET.

  • Run tests

    # from liboffkv/cmake-build-debug directory
    make test

License

Licensed under either of

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

About

liboffkv is a C++ library

Resources

License

Apache-2.0, MIT licenses found

Licenses found

Apache-2.0
LICENSE-APACHE
MIT
LICENSE-MIT

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C++ 83.2%
  • CMake 10.4%
  • C 4.0%
  • Shell 2.4%