Skip to content

Conversation

SheliakLyr
Copy link
Contributor

Fixes #18255

Context

I do not know how many users are impact by this, but as described in related issue, I encountered much higher memory usage after the upgrade to the newer version of gradle (although I might be wrong, problems are very non-deterministic and sometimes happen even on an older version).

I have tracked the issue to the execution history cache. I believe it keeps some fingerprints for task inputs in memory. Those can get very big. In my case, single entry may be as large as a few MB (hunders/thousands of individual filenames).

I think that sizing of this cache should take into account that some keys might be much heavier that others - but let's leave it for the future. For know, I propose to simply add a property for the size of this cache. My problems disappear when I lower the default value from 10000 to 500 (20x!).

This will allow users with similar issues (kind-of-random GC problems) to check if the source is the same.

I haven't found any documentation for the similar property "org.gradle.cache.reserved.mb", so I also decided to skip documenting this one.

Contributor Checklist

  • Review Contribution Guidelines
  • Make sure that all commits are signed off to indicate that you agree to the terms of Developer Certificate of Origin.
  • Make sure all contributed code can be distributed under the terms of the Apache License 2.0, e.g. the code was written by yourself or the original code is licensed under a license compatible to Apache License 2.0.
  • Check "Allow edit from maintainers" option in pull request so that additional changes can be pushed by Gradle team
  • Provide integration tests (under <subproject>/src/integTest) to verify changes from a user perspective
  • Provide unit tests (under <subproject>/src/test) to verify logic
  • Update User Guide, DSL Reference, and Javadoc for public-facing changes
  • Ensure that tests pass sanity check: ./gradlew sanityCheck
  • Ensure that tests pass locally: ./gradlew <changed-subproject>:quickTest

Gradle Core Team Checklist

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation
  • Recognize contributor in release notes

@SheliakLyr
Copy link
Contributor Author

I have noticed a very similar PR: #20006. Maybe my problem can be solved using the property defined in it.

@jbartok jbartok added the in:execution-engine incremental, up-to-date, overlapping outputs label Mar 14, 2022
@big-guy big-guy added the from:contributor PR by an external contributor label Apr 1, 2022
@SheliakLyr
Copy link
Contributor Author

Hello, we have been using a custom build with this fix for about a month now. By "we", I mean ~100 developers. Results are very good. The only problem is the fact that changing the execution history cache size requires manual removal of some files in .gradle/... Not a big deal, since we just had to do this once, but this is an area for further improvements.

@stale
Copy link

stale bot commented Jun 4, 2022

This pull request has been automatically marked as stale because it has not had recent activity. Given the limited bandwidth of the team, it will be closed if no further activity occurs. If you intend to work on this pull request, please ask the team to reopen the PR or push a new PR. Thank you for your contributions.

@stale stale bot added the stale label Jun 4, 2022
@stale
Copy link

stale bot commented Jun 28, 2022

This pull request has been automatically closed due to inactivity. If you are still interested in contributing this, please ensure that it is rebased against the latest branch (usually master), all review comments have been addressed and the build is passing.

@stale stale bot closed this Jun 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

from:contributor PR by an external contributor in:execution-engine incremental, up-to-date, overlapping outputs stale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GC overhead limit exceeded after upgrade from 6.6 to 7.2.

3 participants