Skip to content

Conversation

lmaisons
Copy link
Collaborator

The current implementation of rngfill is prone to false warnings from GCC about array bounds violations. Looking at the code, the implementation naively manipulates both the bytes count and the buffer pointer directly to ensure the trailing memcpy doesn't overrun the buffer. As expressed, there is a data dependency on both fields between loop iterations.

Now, ideally, an optimizing compiler would realize that these dependencies were unnecessary and end up restructuring its intermediate representation into a functionally equivalent form with them absent. However, this is not guaranteed, and even if it does occur, the point at which this happens may well be disjoint from when warning analyses are performed, potentially rendering them more difficult to determine precisely.

In addition, it may also consume a portion of the budget the optimizer has allocated to attempting to improve a translation unit's performance. Given this is a function template which requires context-sensitive instantiation, this code, in the author's experience, would be more prone than most to being inlined, with a decrease in optimization budget corresponding to the effort the optimizer has already expended, having already optimized one or more calling functions. Thus, the scope for impacting the the ultimate quality of the code generated is elevated.

For this change, we rearrange things so that the location and contents of each memcpy can be computed independently, relying on a simple loop iteration counter as the only changing input between iterations.

@lmaisons lmaisons mentioned this pull request Jul 14, 2025
1 task
@bthomee bthomee requested review from Bronek and a1q123456 July 16, 2025 16:18
@Bronek
Copy link
Collaborator

Bronek commented Jul 17, 2025

@lmaisons can you please follow the instructions for signing your commits and amend your commit with signature ?

also, you will need to use clang-format version 18.1.8 to format your changed code (an easy way to install this specific version is pip install clang_format==18.1.8 )

@lmaisons lmaisons force-pushed the rngfill-restructure branch from f5873bc to 002365e Compare July 17, 2025 15:24
@lmaisons
Copy link
Collaborator Author

lmaisons commented Jul 17, 2025

@Bronek Sorry, not used to the flow here yet. Formatted and signed as required.

Copy link
Collaborator

@Bronek Bronek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, just pls. add minor changes.

@Bronek Bronek added the Blocked on requested changes Reviewers have requested changes which must be addressed or responded to label Jul 21, 2025
The current implementation of rngfill is prone to false warnings from
GCC about array bounds violations. Looking at the code, the
implementation naively manipulates both the bytes count and the buffer
pointer directly to ensure the trailing memcpy doesn't overrun the
buffer. As expressed, there is a data dependency on both fields between
loop iterations.

Now, ideally, an optimizing compiler would realize that these
dependencies were unnecessary and end up restructuring its intermediate
representation into a functionally equivalent form with them abscent.
However, the point at which this occurs may be disjoint from when warning
analyises are performed, potentially rendering them more difficult to
determine precisely.

In addition, it may also consume a portion of the budget the optimizer
has allocated to attempting to improve a translation unit's performance.
Given this is a function template which requires context-sensitive
instantiation, this code, in the author's experience, would be more prone
than most to being inlined, with a decrease in optimization budget
corresponding to the effort the optimizer has already expended, having
already optimized one or more calling functions. Thus, the scope for
impacting the the ultimate quality of the code generated is elevated.

For this change, we rearrange things so that the location and contents
of each memcpy can be computed independently, relying on a simple loop
iteration counter as the only changing input between iterations.
@lmaisons lmaisons force-pushed the rngfill-restructure branch from 002365e to c72e18f Compare July 21, 2025 18:06
Copy link

codecov bot commented Jul 21, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.4%. Comparing base (03e46cd) to head (3ba8dba).
Report is 1 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff            @@
##           develop   #5563     +/-   ##
=========================================
- Coverage     78.4%   78.4%   -0.0%     
=========================================
  Files          816     816             
  Lines        71711   71713      +2     
  Branches      8580    8597     +17     
=========================================
- Hits         56242   56224     -18     
- Misses       15469   15489     +20     
Files with missing lines Coverage Δ
include/xrpl/beast/utility/rngfill.h 100.0% <100.0%> (ø)

... and 9 files with indirect coverage changes

Impacted file tree graph

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Bronek Bronek self-requested a review July 22, 2025 08:37
@Bronek Bronek requested a review from vvysokikh1 July 22, 2025 08:42
@Bronek Bronek added Needs second review PR requires at least one more code review approval before it can be merged and removed Blocked on requested changes Reviewers have requested changes which must be addressed or responded to labels Jul 22, 2025
template <class Generator>
void
rngfill(void* buffer, std::size_t bytes, Generator& g)
rngfill(void* const buffer, std::size_t const bytes, Generator& g)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: const in a context of 'std::size_t const bytes' is not really necessary.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's helpful for the optimizer.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it matters in terms of optimization. Compiler is perfectly capable of seeing if the argument changes in the function.

It's more of debatable thing where some people believe it improves readability, others think it's meaningless (see abseil code style for example: https://abseil.io/tips/109)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's true that the optimizer cannot rely on const annotations for correctness. I included it here to communicate to a reader the intent of not updating the field's value in-place in order to provide as much clarity as possible about any mutable data dependencies that arise across basic blocks / loop iterations.

@vvysokikh1 vvysokikh1 removed the Needs second review PR requires at least one more code review approval before it can be merged label Jul 22, 2025
@Bronek
Copy link
Collaborator

Bronek commented Jul 22, 2025

@lmaisons do you think this is ready to merge ?

@lmaisons
Copy link
Collaborator Author

@lmaisons do you think this is ready to merge ?

Yes.

@Bronek Bronek added the Ready to merge *PR author* thinks it's ready to merge. Has passed code review. Perf sign-off may still be required. label Jul 22, 2025
@bthomee bthomee merged commit 6090965 into XRPLF:develop Jul 22, 2025
25 checks passed
@lmaisons lmaisons deleted the rngfill-restructure branch July 22, 2025 15:49
This was referenced Aug 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready to merge *PR author* thinks it's ready to merge. Has passed code review. Perf sign-off may still be required.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants