Skip to content

feat(ss-bench): support --scans option #503

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Feb 24, 2022
Merged

feat(ss-bench): support --scans option #503

merged 5 commits into from
Feb 24, 2022

Conversation

@Sunt-ing
Copy link
Contributor

@Sunt-ing Sunt-ing commented Feb 23, 2022 •
edited
Loading

What's changed and what's your intention?

  • feat: support --scans option. It seems that RocksDB db_bench has no such option, but I think this is important and thus finally I think I need to add it.
  • fix: fix batches size estimation
  • docs: Add some doc.

Example output:

 ~/code/risingwave-dev/rust   sunt_scans_param  cargo run --bin ss-bench -- --benchmarks "writebatch,getrandom,getseq,prefixscanrandom,deleterandom" --batch-size 1000 --reads 1000 --scans 500 --writes 200000 --deletes 100000 --statistics --concurrency-num 4 --seed 4
    Finished dev [unoptimized + debuginfo] target(s) in 0.35s
     Running `target/debug/ss-bench --benchmarks writebatch,getrandom,getseq,prefixscanrandom,deleterandom --batch-size 1000 --reads 1000 --scans 500 --writes 200000 --deletes 100000 --statistics --concurrency-num 4 --seed 4`
Configurations before preprocess:
 Opts { store: "in-memory", table_size_mb: 256, block_size_kb: 64, bloom_false_positive: 0.1, checksum_algo: "crc32c", benchmarks: "writebatch,getrandom,getseq,prefixscanrandom,deleterandom", concurrency_num: 4, num: 1000000, deletes: 100000, reads: 1000, scans: 500, writes: 2000, batch_size: 1000, key_size: 16, key_prefix_size: 5, keys_per_prefix: 10, value_size: 100, seed: 4, statistics: true }
Configurations after preprocess:
 Opts { store: "in-memory", table_size_mb: 256, block_size_kb: 64, bloom_false_positive: 0.1, checksum_algo: "crc32c", benchmarks: "writebatch,getrandom,getseq,prefixscanrandom,deleterandom", concurrency_num: 4, num: 1000000, deletes: 100000, reads: 1000, scans: 500, writes: 2000, batch_size: 1000, key_size: 16, key_prefix_size: 5, keys_per_prefix: 10, value_size: 100, seed: 4, statistics: true }

    writebatch
      latency:
        min: 2.763 milli sec,
        mean: 3.937 milli sec,
        p50: 5.111 milli sec,
        p90: 5.111 milli sec,
        p99: 5.111 milli sec,
        max: 5.111 milli sec,
        std_dev: 1174267.000;
      KV ingestion OPS: 362925  23227210 bytes/sec

    getrandom
      latency:
        min: 29.850 micro sec,
        mean: 34.300 micro sec,
        p50: 32.280 micro sec,
        p90: 36.017 micro sec,
        p99: 69.406 micro sec,
        max: 144.284 micro sec,
        std_dev: 8581.633;
      QPS: 112758  11275880 bytes/sec

    getseq
      latency:
        min: 4.391 micro sec,
        mean: 36.003 micro sec,
        p50: 32.099 micro sec,
        p90: 47.605 micro sec,
        p99: 103.599 micro sec,
        max: 133.655 micro sec,
        std_dev: 12361.928;
      QPS: 107887  10788777 bytes/sec

    prefixscanrandom
      latency:
        min: 61.321 micro sec,
        mean: 78.482 micro sec,
        p50: 68.956 micro sec,
        p90: 106.370 micro sec,
        p99: 202.051 micro sec,
        max: 239.240 micro sec,
        std_dev: 24880.073;
      QPS: 98225  1178711 bytes/sec

    deleterandom
      latency:
        min: 516.475 micro sec,
        mean: 995.962 micro sec,
        p50: 581.285 micro sec,
        p90: 2.303 milli sec,
        p99: 6.026 milli sec,
        max: 6.026 milli sec,
        std_dev: 1004984.988;
      KV ingestion OPS: 3911181  196843191 bytes/sec
STATISTICS:
state_store_get_latency P50 : 0.000026525198938992044 P95 : 0.000057211538461538465 P99 : 0.00009567307692307693 P100 : 0.00025 COUNT : 2000 SUM : 0.063455112
state_store_get_key_size P50 : 35 P95 : 48.5 P99 : 49.7 P100 : 50 COUNT : 2000 SUM : 42000
state_store_get_value_size P50 : 35 P95 : 48.5 P99 : 49.7 P100 : 50 COUNT : 2000 SUM : 64000
state_store_get_counts COUNT : 2000
state_store_get_snapshot_latency P50 : 0 P95 : 0 P99 : 0 P100 : 0 COUNT : 0 SUM : 0
state_store_range_scan_counts COUNT : 500
state_store_reverse_range_scan_counts COUNT : 0
state_store_batched_write_counts COUNT : 102
state_store_batched_write_tuple_counts COUNT : 80638
state_store_batched_write_latency P50 : 0.00081875 P95 : 0.0041666666666666675 P99 : 0.008333333333333335 P100 : 0.010000000000000002 COUNT : 102 SUM : 0.107110285
state_store_batched_write_size P50 : 50 P95 : 95.09803921568627 P99 : 99.01960784313725 P100 : 100 COUNT : 102 SUM : 2448
state_store_batch_write_build_table_latency P50 : 0 P95 : 0 P99 : 0 P100 : 0 COUNT : 0 SUM : 0
state_store_batch_write_add_l0_ssts_latency P50 : 0 P95 : 0 P99 : 0 P100 : 0 COUNT : 0 SUM : 0
state_store_iter_counts COUNT : 0
state_store_iter_next_counts COUNT : 0
state_store_iter_seek_latency P50 : 0 P95 : 0 P99 : 0 P100 : 0 COUNT : 0 SUM : 0
state_store_iter_next_latency P50 : 0 P95 : 0 P99 : 0 P100 : 0 COUNT : 0 SUM : 0
state_store_pin_version_counts COUNT : 0
state_store_unpin_version_counts COUNT : 0
state_store_pin_snapshot_counts COUNT : 0
state_store_unpin_snapshot_counts COUNT : 0
state_store_add_tables_counts COUNT : 0
state_store_get_new_table_id_counts COUNT : 0
state_store_get_compaction_task_counts COUNT : 0
state_store_report_compaction_task_counts COUNT : 0
state_store_pin_version_latency P50 : 0 P95 : 0 P99 : 0 P100 : 0 COUNT : 0 SUM : 0
state_store_unpin_version_latency P50 : 0 P95 : 0 P99 : 0 P100 : 0 COUNT : 0 SUM : 0
state_store_pin_snapshot_latency P50 : 0 P95 : 0 P99 : 0 P100 : 0 COUNT : 0 SUM : 0
state_store_unpin_snapshot_latency P50 : 0 P95 : 0 P99 : 0 P100 : 0 COUNT : 0 SUM : 0
state_store_add_tables_latency P50 : 0 P95 : 0 P99 : 0 P100 : 0 COUNT : 0 SUM : 0
state_store_get_new_table_id_latency P50 : 0 P95 : 0 P99 : 0 P100 : 0 COUNT : 0 SUM : 0
state_store_get_compaction_task_latency P50 : 0 P95 : 0 P99 : 0 P100 : 0 COUNT : 0 SUM : 0
state_store_report_compaction_task_latency P50 : 0 P95 : 0 P99 : 0 P100 : 0 COUNT : 0 SUM : 0

PLEASE DO NOT LEAVE THIS EMPTY !!!

Please explain IN DETAIL what the changes are in this PR and why they are needed:

  • Summarize your change (mandatory)
  • How does this PR work? Need a brief introduction for the changed logic (optional)
  • Describe clearly one logical change and avoid lazy messages (optional)
  • Describe any limitations of the current code (optional)

Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests

Refer to a related PR or issue link (optional)

@Sunt-ing Sunt-ing requested a review from lmatz February 23, 2022 09:00
@github-actions github-actions bot added the type/feature Type: New feature. label Feb 23, 2022
@codecov
Copy link

codecov bot commented Feb 23, 2022 •
edited
Loading

Codecov Report

Merging #503 (a7e4c68) into main (ed6330a) will increase coverage by 0.02%.
The diff coverage is 0.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##               main     #503      +/-   ##
============================================
+ Coverage     72.73%   72.76%   +0.02%     
  Complexity     2709     2709              
============================================
  Files           876      876              
  Lines         49079    49086       +7     
  Branches       1557     1557              
============================================
+ Hits          35700    35716      +16     
+ Misses        12568    12559       -9     
  Partials        811      811
Flag Coverage Δ
java 63.40% <ø> (ø)
rust 76.70% <0.00%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
rust/bench/ss_bench/main.rs 0.00% <0.00%> (ø)
...st/bench/ss_bench/operations/prefix_scan_random.rs 0.00% <0.00%> (ø)
rust/bench/ss_bench/operations/write_batch.rs 0.00% <0.00%> (ø)
rust/bench/ss_bench/utils/workload.rs 0.00% <0.00%> (ø)
rust/common/src/types/ordered_float.rs 25.49% <0.00%> (-0.34%) ⬇️
rust/stream/src/executor/barrier_align.rs 78.78% <0.00%> (ø)
rust/meta/src/stream/meta.rs 48.78% <0.00%> (+7.31%) ⬆️
rust/meta/src/barrier/info.rs 92.30% <0.00%> (+7.69%) ⬆️
rust/meta/src/model/stream.rs 78.57% <0.00%> (+14.28%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ed6330a...a7e4c68. Read the comment docs.

@Sunt-ing Sunt-ing merged commit 1b16ec6 into main Feb 24, 2022
@Sunt-ing Sunt-ing deleted the sunt_scans_param branch February 24, 2022 04:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lmatz lmatz lmatz approved these changes

Assignees

No one assigned

Labels

type/feature Type: New feature.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Sunt-ing @lmatz