-
Notifications
You must be signed in to change notification settings - Fork 14.7k
MINOR: Update to Gradle 6.5 and tweak build jvm config #8751
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@omkreddy I'll rerun the numbers with 6.5 final tomorrow morning, but this should be ready for review otherwise. |
@omkreddy Updated the PR description and made a few more changes. |
JDK 8/Scala 2.12 had an out of memory issue. |
retest this please |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ijuma Thanks for the PR. LGTM.
Build passed. |
Cherry picked this to 2.6 as well since it's low risk and fixes a compilation speed regression. |
Gradle 6.5 includes a fix for gradle/gradle#12866, which affects the performance of Scala compilation. I profiled the scalac build with async profiler and 54% of the time was on GC even after the Gradle upgrade (it was more than 60% before), so I switched to the throughput GC (GC latency is less important for batch builds) and it was reduced to 38%. I also centralized the jvm configuration in `build.gradle` and simplified it a bit by removing the minHeapSize configuration from the test tasks. On my desktop, the time to execute clean builds with no cached Gradle daemon was reduced from 127 seconds to 97 seconds. With a cached daemon, it was reduced from 120 seconds to 88 seconds. The performance regression when we upgraded to Gradle 6.x was 27 seconds with a cached daemon (#7677 (comment)), so it should be fixed now. Gradle 6.4 with no cached daemon: ``` BUILD SUCCESSFUL in 2m 7s 115 actionable tasks: 112 executed, 3 up-to-date ./gradlew clean compileScala compileJava compileTestScala compileTestJava 1.15s user 0.12s system 0% cpu 2:08.06 total ``` Gradle 6.4 with cached daemon: ``` BUILD SUCCESSFUL in 2m 0s 115 actionable tasks: 111 executed, 4 up-to-date ./gradlew clean compileScala compileJava compileTestScala compileTestJava 0.95s user 0.10s system 0% cpu 2:01.42 total ``` Gradle 6.5 with no cached daemon: ``` BUILD SUCCESSFUL in 1m 46s 115 actionable tasks: 111 executed, 4 up-to-date ./gradlew clean compileScala compileJava compileTestScala compileTestJava 1.27s user 0.12s system 1% cpu 1:47.71 total ``` Gradle 6.5 with cached daemon: ``` BUILD SUCCESSFUL in 1m 37s 115 actionable tasks: 111 executed, 4 up-to-date ./gradlew clean compileScala compileJava compileTestScala compileTestJava 1.02s user 0.10s system 1% cpu 1:38.31 total ``` This PR with no cached Gradle daemon: ``` BUILD SUCCESSFUL in 1m 37s 115 actionable tasks: 81 executed, 34 up-to-date ./gradlew clean compileScala compileJava compileTestScala compileTestJava 1.27s user 0.10s system 1% cpu 1:38.70 total ``` This PR with cached Gradle daemon: ``` BUILD SUCCESSFUL in 1m 28s 115 actionable tasks: 111 executed, 4 up-to-date ./gradlew clean compileScala compileJava compileTestScala compileTestJava 1.02s user 0.10s system 1% cpu 1:29.35 total ``` Reviewers: Manikumar Reddy <[email protected]>, Chia-Ping Tsai <[email protected]>
…use_all_dns_ips-as-default * apache-github/trunk: KAFKA-9788; Use distinct names for transaction and group load time sensors (#8784) KAFKA-9514; The protocol generator generated useless condition when a field is made nullable and flexible version is used (#8793) MINOR: Update to Gradle 6.5 and tweak build jvm config (#8751) MINOR: Upgrade spotbugs and spotbugsPlugin (#8790) KAFKA-10089 The stale ssl engine factory is not closed after reconfigure (#8792) KAFKA-10080; Fix race condition on txn completion which can cause duplicate appends (#8782) KAFKA-10084: Fix EosTestDriver end offset (#8785) KAFKA-10083: fix failed testReassignmentWithRandomSubscriptionsAndChanges tests (#8786)
Gradle 6.5 includes a fix for gradle/gradle#12866, which
affects the performance of Scala compilation.
I profiled the scalac build with async profiler and 54% of the time was on GC
even after the Gradle upgrade (it was more than 60% before), so I switched to
the throughput GC (GC latency is less important for batch builds) and it
was reduced to 38%.
I also centralized the jvm configuration in
build.gradle
and simplified it a bitby removing the minHeapSize configuration from the test tasks.
On my desktop, the time to execute clean builds with no cached Gradle daemon
was reduced from 127 seconds to 97 seconds. With a cached daemon, it was
reduced from 120 seconds to 88 seconds. The performance regression when
we upgraded to Gradle 6.x was 27 seconds with a cached daemon
(#7677 (comment)), so it
should be fixed now.
Gradle 6.4 with no cached daemon:
Gradle 6.4 with cached daemon:
Gradle 6.5 with no cached daemon:
Gradle 6.5 with cached daemon:
This PR with no cached Gradle daemon:
This PR with cached Gradle daemon:
Committer Checklist (excluded from commit message)