[storage/adb/any] create a new benchmark for the variable anydb #1388
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR creates a new benchmark for the variable anydb that measures initialization performance with varying-sized vectors instead of fixed digests. The implementation follows the existing fixeddb benchmark pattern but uses Vec<u8> values of variable length (7-21 bytes) instead of fixed digest values.
Key changes:
- Created a new
variable_init.rsbenchmark module for testing variable-sized data - Renamed the existing benchmark function from
bench_any_inittobench_fixed_initfor clarity - Updated the benchmark runner to include both fixed and variable benchmarks
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| storage/src/adb/benches/variable_init.rs | New benchmark module implementing variable-sized vector testing |
| storage/src/adb/benches/fixed_init.rs | Renamed benchmark function for consistency |
| storage/src/adb/benches/bench.rs | Updated module imports and criterion_main to include both benchmarks |
015ebfd to
1b5f925
Compare
| let mut rng = StdRng::seed_from_u64(42); | ||
| for i in 0u64..num_elements { | ||
| let k = hash(&i.to_be_bytes()); | ||
| let v = vec![(rng.next_u32() % 255) as u8; ((rng.next_u32() % 14) + 8) as usize]; |
There was a problem hiding this comment.
Would it make more sense to have the size of this be up to ~32 bytes (for a better parity check with fixed)?
There was a problem hiding this comment.
Changed values to have avg. of 32 bytes
| let runner = tokio::Runner::new(cfg.clone()); | ||
| for elements in [NUM_ELEMENTS, NUM_ELEMENTS * 2] { | ||
| for operations in [NUM_OPERATIONS, NUM_OPERATIONS * 2] { | ||
| info!(elements, operations, "benchmarking any init",); |
| let runner = tokio::Runner::new(cfg.clone()); | ||
| for elements in [NUM_ELEMENTS, NUM_ELEMENTS * 2] { | ||
| for operations in [NUM_OPERATIONS, NUM_OPERATIONS * 2] { | ||
| info!(elements, operations, "benchmarking any init",); |
There was a problem hiding this comment.
nit: benchmarking any::variable?
|
|
||
| /// Benchmark the initialization of a large randomly generated any db. | ||
| fn bench_variable_init(c: &mut Criterion) { | ||
| tracing_subscriber::fmt().try_init().ok(); |
There was a problem hiding this comment.
nit: in other benchmarks, I don't perform any logging?
(I suppose it is preferred to do so?)
There was a problem hiding this comment.
I added this because startup was really slow (creating the massive DB to then replay) and wanted to see it making some progress...
There was a problem hiding this comment.
(This is outside the part being timed)
1b5f925 to
7bb592a
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## main #1388 +/- ##
=======================================
Coverage 91.34% 91.34%
=======================================
Files 265 265
Lines 66726 66726
=======================================
+ Hits 60950 60952 +2
+ Misses 5776 5774 -2 see 1 file with indirect coverage changes Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
The benchmark follows the implementation of the any::Fixed init benchmark, only inserting varying sized vectors instead of fixed digests.
Because reads from the variable journal are not currently buffered, this benchmark runs slowly (3-7 seconds per iteration depending on the parameters). It will allow us to measure the impact of buffering in #1384