Float Compressions and Original File Format for databases
Currently under developments.
Configure target cpu by setting envvars when building:
AVX2:
$ RUSTFLAGS="-C target-cpu=haswell" TARGET_CPU=haswell cargo build --releaseAVX512: (if your CPU supports)
$ RUSTFLAGS="-C target-cpu=native" TARGET_CPU=native cargo build --releaseEnsure that your datasets are in the DATA_DICT
# create binary files from csv
$ cargo run --bin csv2bin --release -- "DATA_DICT/*"
$ cd experimenter
# create directory for saving experiment checkpoints
$ mkdir save compressor_configs data/filter_config
$ sudo ../target/release/experimenter all -c compressor_configs --exact-precision -f DATA_DICT/filter_config -b DATA_DICT/binary --create-config --n 10 --in-memory -s saveResults will be saved in DATA_DICT/result
$ cd experimenter
$ sudo ../target/release/experimenter -i DATA_DICT/UCRArchive_2018/ ucr2018 -o DATA_DICTResults will be saved in DATA_DICT/result
$ cd experimenter
$ cargo build --release --features=cuda # Please also specify your RUSTFLAGS and TARGET_CPU
$ sudo ../target/release/experimenter matrix-cuda -o OUTDIR$ cd experimenter
$ sudo ../target/release/experimenter -i DATA_DICT embedding -o DATA_DICTResults will be saved in DATA_DICT/result
$ cd tools
$ uv run gen-tpch.py # generate files on tools/tpch_data/
$ cd experimenter
$ sudo ../target/release/experimenter tpch -i DATA_DIR -o OUT_DIRResults will be saved in OUT_DIR/tpch
See tools/README.md.
This project is licensed under the MIT License - see the LICENSE file for details.
By contributing to this project, you agree that your contributions will be licensed under the MIT License, unless you explicitly state otherwise in your submission.