Skip to content

Conversation

danlaine
Copy link
Collaborator

@danlaine danlaine commented Jul 16, 2025

This PR resolves #1190.

  • Factors out struct SyncTarget
  • Adds update_receiver channel to client state which receives sync target updates
  • Corresponding sender is (optionally) provided when creating sync client
  • Client checks for updated target before each step
  • Targets may not move "backward" -- the hash must be updated and the bounds must move upward.

@danlaine danlaine self-assigned this Jul 16, 2025
Base automatically changed from danlaine/sync-lower-bound to main July 16, 2025 14:53
@danlaine danlaine force-pushed the danlaine/sync-dynamic-root branch from d77c08f to 3dbc2dd Compare July 16, 2025 20:34
@danlaine danlaine requested a review from Copilot July 16, 2025 20:58
@danlaine danlaine changed the title [WIP] [adb] [sync] Make sync target dynamic [adb] [sync] Make sync target dynamic Jul 16, 2025
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements dynamic sync targets for the ADB sync client, allowing the sync target (hash and operation bounds) to be updated during the synchronization process. The key changes include factoring out a SyncTarget struct to encapsulate sync parameters, adding an update channel mechanism for receiving target changes, and implementing validation to ensure targets only move "forward" (bounds increase and hash changes).

  • Introduces SyncTarget struct to group hash and operation bounds
  • Adds channel-based mechanism for receiving sync target updates during sync
  • Implements validation to prevent backward movement of sync targets

Reviewed Changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 1 comment.

File Description
storage/src/adb/any/sync/mod.rs Adds SyncTarget struct, update channel types, new error variants, and sync_with_updates function
storage/src/adb/any/sync/client.rs Refactors Config to use SyncTarget, adds update handling logic, and implements target validation with comprehensive tests
Comments suppressed due to low confidence (2)

storage/src/adb/any/sync/client.rs:263

  • [nitpick] The variable name update_ with trailing underscore is unconventional. Consider using a more descriptive name like new_update or pending_update.
            while let Ok(Some(update_)) = receiver.try_next() {

storage/src/adb/any/sync/client.rs:591

  • [nitpick] Empty line added in the middle of imports section. This should be removed to maintain consistent formatting.
    use super::*;

@danlaine danlaine marked this pull request as ready for review July 16, 2025 21:53
}
}

#[cfg(test)]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could compile this outside tests, too, but we only use this in tests

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On further thought, all of these implementations are only used in tests.

I removed the #[cfg(test)] here for consistency. Should all of these implementations be under a #[cfg(test)]?

Copy link

codecov bot commented Jul 17, 2025

Codecov Report

Attention: Patch coverage is 95.37815% with 22 lines in your changes missing coverage. Please review.

Project coverage is 91.16%. Comparing base (86db8e7) to head (6b73352).
Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
storage/src/adb/any/sync/client.rs 95.01% 22 Missing ⚠️
@@            Coverage Diff             @@
##             main    #1281      +/-   ##
==========================================
+ Coverage   91.13%   91.16%   +0.03%     
==========================================
  Files         246      246              
  Lines       60746    61157     +411     
==========================================
+ Hits        55360    55753     +393     
- Misses       5386     5404      +18     
Files with missing lines Coverage Δ
storage/src/adb/any/mod.rs 99.12% <100.00%> (-0.01%) ⬇️
storage/src/adb/any/sync/mod.rs 100.00% <ø> (ø)
storage/src/adb/any/sync/resolver.rs 83.67% <100.00%> (+10.34%) ⬆️
storage/src/adb/any/sync/client.rs 92.48% <95.01%> (+1.70%) ⬆️

... and 2 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 86db8e7...6b73352. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

lower_bound_ops: oldest_retained_loc,
upper_bound_ops: latest_op_loc,
target: SyncTarget {
hash: target_hash,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

root might be a better name if this is indeed some root hash? (i use root implicitly for "root_hash" elsewhere)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also been trying to use "digest" instead of "hash" (though probably haven't updated all occurrences, as was originally using "hash" throughout.)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per Patrick's request, I've renamed all "root_hash" and "root_digest" to just "root" in variable and function names in #1299

log_size,
old_bounds = ?config.target,
new_bounds = ?new_target,
"Log is stale, reinitializing"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

super nitty question but do we want to standardize on capitalizing the first word of any log line? I think I've been leaving it all lowercase unless it's really a full sentence with periods.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I've also been doing all lowercase)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, you're right. #1295 changes the logs in adb to start with lowercase.

Copy link
Contributor

@patrick-ogrady patrick-ogrady left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@patrick-ogrady patrick-ogrady merged commit ebcc30a into main Jul 21, 2025
33 checks passed
@patrick-ogrady patrick-ogrady deleted the danlaine/sync-dynamic-root branch July 21, 2025 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[storage::adb::sync] Allow sync target to change mid-sync

3 participants