Deduplicate configuration cache report problems #36047

abstratt · 2025-12-16T14:32:53Z

Issue: #36046 (see reproducer there)

Without the fix:

5000 problems were found storing the configuration cache, only the first 4096 were considered, 1 of which seems unique.
- Build file 'child/build.gradle': line 2: Project ':child' cannot dynamically look up a property in the parent project ':'

Report size: 107M

With the fix:

5000 problems were found storing the configuration cache, 1 of which seems unique.
- Build file 'child/build.gradle': line 2: Project ':child' cannot dynamically look up a property in the parent project ':'

Report size: 153K

Reviewing cheatsheet

Before merging the PR, comments starting with

❌ ❓must be fixed
🤔 💅 should be fixed
💭 may be fixed
🎉 celebrate happy things

abstratt · 2025-12-16T17:43:40Z

.../groovy/org/gradle/internal/cc/impl/ConfigurationCacheProblemReportingIntegrationTest.groovy

            withProblem("Task `:problems` of type `BrokenTask`: cannot serialize object of type 'org.gradle.api.internal.project.DefaultProject', a subtype of 'org.gradle.api.Project', as these are not supported with the configuration cache.")
            totalProblemsCount = 6
-            problemsWithStackTraceCount = 2
+            problemsWithStackTraceCount = 1


these are unique counts now

abstratt · 2025-12-16T17:44:42Z

...ion-cache/src/main/kotlin/org/gradle/internal/cc/impl/problems/ConfigurationCacheProblems.kt

        addNotReportedDegradingTasks()
        addDegradingFeatures()
        val summary = summarizer.get()
-        val hasNoProblemsForConsole = summary.reportableProblemCount == 0


I got confused by reportableProblemCount - it really means a count of problems we report to the console

abstratt · 2025-12-16T18:03:27Z

...he/src/main/kotlin/org/gradle/internal/cc/impl/problems/ConfigurationCacheProblemsSummary.kt

    val lock = ReentrantLock()

    fun get(): Summary = lock.withLock {
        Summary(


Many parameters here have same type, wanted to make the data flows more explicit

abstratt · 2025-12-16T18:04:11Z

...he/src/main/kotlin/org/gradle/internal/cc/impl/problems/ConfigurationCacheProblemsSummary.kt

+            totalProblemCount = totalProblemCount,
+            uniqueProblemCount = problemCauses.size,
+            deferredProblemCount = deferredProblemCount,
+            consoleProblemCount = totalProblemCount - suppressedSilentlyProblemCount,


This is an actual change

abstratt · 2025-12-16T18:05:15Z

...he/src/main/kotlin/org/gradle/internal/cc/impl/problems/ConfigurationCacheProblemsSummary.kt

+            uniqueProblemCount = problemCauses.size,
+            deferredProblemCount = deferredProblemCount,
+            consoleProblemCount = totalProblemCount - suppressedSilentlyProblemCount,
+            consoleProblemCauses = ImmutableMap.copyOf(problemCauses.filterValues { it != ProblemSeverity.SuppressedSilently }),


So is this. Since we only care for problem causes we can show on the console, decided to filter it here, and reflect that in the property name

abstratt · 2025-12-16T18:06:27Z

...he/src/main/kotlin/org/gradle/internal/cc/impl/problems/ConfigurationCacheProblemsSummary.kt

                overflowed = true
                return false
            }
-            if (severity != ProblemSeverity.SuppressedSilently) {


Before, we would not apply deduplication to problems with this severity, now we do

🤔 Do we cover this by integTest?

We do with a unit test:

gradle/platforms/core-configuration/configuration-cache/src/test/kotlin/org/gradle/internal/cc/impl/problems/ConfigurationCacheProblemsSummaryTest.kt

Line 124 in 1ac66b7

assertThat(accepted, equalTo(unique))

Also, integration tests had to be updated to reflect this, including a smoke test.

abstratt · 2025-12-16T18:07:47Z

...he/src/main/kotlin/org/gradle/internal/cc/impl/problems/ConfigurationCacheProblemsSummary.kt

-    fun onIncompatibleFeature() {
+    fun onIncompatibleFeature(problem: PropertyProblem) {
        lock.withLock {
+            onProblem(problem, ProblemSeverity.SuppressedSilently)


For incompatible features, we were adding a problem to the report without letting the summarizer know - now we do report the problem so it is reflected on totalProblemCount and suppressedSilentlyProblemCount.

abstratt · 2025-12-16T18:10:03Z

...he/src/main/kotlin/org/gradle/internal/cc/impl/problems/ConfigurationCacheProblemsSummary.kt

+     * Returns whether problem was accepted.
     */
    private
    fun recordProblemCause(problem: PropertyProblem, severity: ProblemSeverity): Boolean {


Before, this may return false even if the new problem upgraded the severity of a previous one

abstratt · 2025-12-16T18:21:27Z

...ovy/org/gradle/integtests/fixtures/configurationcache/ConfigurationCacheReportFixture.groovy


            assertThat(
                "HTML report JS model has wrong number of total problem(s)",
-                numberOfProblems(),


this was relying on number of problems in the data - now that we deduplicate, we need to query the totalProblemCount slot instead

abstratt · 2025-12-16T18:22:12Z

...ovy/org/gradle/integtests/fixtures/configurationcache/ConfigurationCacheReportFixture.groovy

        }

        private void assertProblemsHtmlReport(HasConfigurationCacheProblemsSpec spec) {
-            def totalProblemCount = spec.totalProblemsCount ?: spec.uniqueProblems.size()


this meant that if totalProblemsCount == 0 we would ignore it.

❓ Not sure, why?

Because 0 ?: something is always something (0 is the numerical falsy value)

abstratt · 2025-12-16T18:23:06Z

...ovy/org/gradle/integtests/fixtures/configurationcache/ConfigurationCacheReportFixture.groovy

            )

            if (spec.checkReportProblems) {
-                def problemMessages = problemMessages().unique()


no need to remove duplicates here

abstratt · 2025-12-16T18:32:13Z

...in/groovy/org/gradle/integtests/fixtures/configurationcache/ConfigurationCacheFixture.groovy


        int getProblemsWithStackTrace() {
-            return problems.inject(0) { a, b -> a + (b.hasStackTrace ? b.count : 0) }
+            return problems.inject(0) { a, b -> a + (b.hasStackTrace ? 1 : 0) }


This count is unique now

bot-gradle · 2025-12-16T21:15:56Z

The following builds have failed:

Ready for Nightly (Trigger) (Check)
- Build Scan

Isue: #36047

abstratt · 2025-12-16T23:13:11Z

...-test/src/smokeTest/groovy/org/gradle/smoketests/GradleBuildIsolatedProjectsSmokeTest.groovy

                "Project ':' cannot access 'Project.plugins' functionality on subprojects via 'allprojects'",
                "Project ':' cannot access 'Project.extensions' functionality on subprojects via 'allprojects'",
            )
-            // maximum number of problems we collect (should be 86520)


This was what made us notice the issue with lack of deduplication. I think 86520 was what I saw with a previous version of the init script used by this test which would fail to schedule some tasks. When that was addressed, and more tasks scheduled grew, so did the number of violations.

abstratt · 2025-12-16T23:14:38Z

@bot-gradle test RFN

bot-gradle · 2025-12-17T00:08:12Z

The following builds have passed:

Ready for Nightly (Trigger) (Check)
- Build Scan

Issue: #36046

6hundreds · 2025-12-17T09:49:41Z

...oovy/org/gradle/internal/cc/impl/ConfigurationCacheGracefulDegradationIntegrationTest.groovy

-            totalProblemsCount = 2
+            // gracefully degraded task appears in the report
+            withProblem(INVOCATION_OF_TASK_PROJECT_AT_EXECUTION_TIME)


🤔 Why don't we check total problems count anymore?

🤔 I wonder, why do we need to add the same withProblem(xxx)? The deal is withProblems adds a problem to uniqueProblems set, so I'd assume that it should be unique?

We do (it is always checked), but since it is the same as the number of unique problems, it is not necessary to explicitly state it.

The set of expected unique problems is actually a list. Each expected unique problem must match each actual unique problem message, in the same position they are expected to appear. The matching is based on message alone, not location, but uniqueness considers both message and location, so the same kind of problem (with identical messages) may appear multiple times.

I think Mike's PR will allow us to match locations as well.

6hundreds · 2025-12-17T10:36:57Z

...he/src/main/kotlin/org/gradle/internal/cc/impl/problems/ConfigurationCacheProblemsSummary.kt

+     * Builds the console feedback string for configuration cache problems.
+     *
+     * Rules:
+     * - If there are reportable problems, print a summary header and a curated list of up to [MAX_CONSOLE_PROBLEMS].


🤔 Should we use "console" instead of "reportable" ?

6hundreds · 2025-12-17T10:49:00Z

...he/src/main/kotlin/org/gradle/internal/cc/impl/problems/ConfigurationCacheProblemsSummary.kt

-                if (reportableProblemCauseCount > MAX_CONSOLE_PROBLEMS) {
-                    appendLine("plus ${reportableProblemCauseCount - MAX_CONSOLE_PROBLEMS} more problems. Please see the report for details.")
+                if (uniqueProblemCount > MAX_CONSOLE_PROBLEMS) {
+                    appendLine("plus ${uniqueProblemCount - MAX_CONSOLE_PROBLEMS} more problems. Please see the report for details.")


❓ Did we change the logic here?
Before, we were checking reportableProblemCauseCount > MAX_CONSOLE_PROBLEMS, and then print "plus ${reportableProblemCauseCount - MAX_CONSOLE_PROBLEMS} more problems."
Now we do same, but instead of reportableProblems we use uniqueProblems, which includes suppressed silently problems

Good catch, thanks! I will add a test and fix this.

6hundreds · 2025-12-17T11:10:31Z

...rc/test/kotlin/org/gradle/internal/cc/impl/problems/ConfigurationCacheProblemsSummaryTest.kt

+const val REPORT_URL: String = "<<REPORT_URL_PLACEHOLDER>>"
+
+const val ACTION: String = "<<ACTION_PLACEHOLDER>>"


❓ How these placeholder get populated?

They are used as fixed values for report URL and action strings, respectively, and make the actual output predictable.

6hundreds · 2025-12-17T11:14:49Z

...rc/test/kotlin/org/gradle/internal/cc/impl/problems/ConfigurationCacheProblemsSummaryTest.kt

-        val subject = ConfigurationCacheProblemsSummary(maxCollectedProblems = 2)
+        val subject = ConfigurationCacheProblemsSummary(maxCollectedProblems = 3)
+
+        // causes for unique problems are all collected


🤔 Not sure I understand the comment

I will try to improve it in my next commit

6hundreds · 2025-12-17T11:16:30Z

...rc/test/kotlin/org/gradle/internal/cc/impl/problems/ConfigurationCacheProblemsSummaryTest.kt

+        assertThat(
+            subject.get().uniqueProblemCount,
+            equalTo(1)
+        )


💅 Maybe `assert(subject.get().uniqueProblemCount == 1) is more concise?

Indeed, or assertEquals... I tried to stay with the flavor of assertions in place as much as possible

There is no assertEquals on the classpath there tho :(

6hundreds

Great job! Left some comments

abstratt added this to the 9.4.0 RC1 milestone Dec 16, 2025

This comment has been minimized.

Sign in to view

abstratt force-pushed the rchaves/master/max-unique-problems branch from 03fa21f to 0f1ecef Compare December 16, 2025 17:42

abstratt self-assigned this Dec 16, 2025

abstratt commented Dec 16, 2025

View reviewed changes

abstratt force-pushed the rchaves/master/max-unique-problems branch from 0f1ecef to 0b30add Compare December 16, 2025 17:58

abstratt commented Dec 16, 2025

View reviewed changes

abstratt force-pushed the rchaves/master/max-unique-problems branch 2 times, most recently from be5eed3 to fad4665 Compare December 16, 2025 18:19

abstratt commented Dec 16, 2025

View reviewed changes

This comment has been minimized.

Sign in to view

abstratt commented Dec 16, 2025

View reviewed changes

abstratt force-pushed the rchaves/master/max-unique-problems branch from fad4665 to 997b249 Compare December 16, 2025 18:37

abstratt marked this pull request as ready for review December 16, 2025 18:39

abstratt requested review from a team as code owners December 16, 2025 18:39

abstratt requested review from 6hundreds and bamboo and removed request for a team December 16, 2025 18:39

cobexer removed the request for review from a team December 16, 2025 18:40

This comment has been minimized.

Sign in to view

abstratt added a commit that referenced this pull request Dec 16, 2025

Update expectation now that we no longer give up

ad6ef3d

Isue: #36047

abstratt linked an issue Dec 16, 2025 that may be closed by this pull request

Configuration cache report threshold for unique problems is triggered by non-unique problems #36046

Open

abstratt commented Dec 16, 2025

View reviewed changes

This comment has been minimized.

Sign in to view

Deduplicate configuration cache report problems

1ac66b7

Issue: #36046

abstratt force-pushed the rchaves/master/max-unique-problems branch from ad6ef3d to 1ac66b7 Compare December 17, 2025 03:01

6hundreds reviewed Dec 17, 2025

View reviewed changes

6hundreds requested changes Dec 17, 2025

View reviewed changes

Fix issue with MAX_CONSOLE_PROBLEMS vs suppressed problems

a739519

		const val REPORT_URL: String = "<<REPORT_URL_PLACEHOLDER>>"

		const val ACTION: String = "<<ACTION_PLACEHOLDER>>"

Deduplicate configuration cache report problems #36047

Are you sure you want to change the base?

Deduplicate configuration cache report problems #36047

Conversation

abstratt commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Without the fix:

With the fix:

Reviewing cheatsheet

Uh oh!

This comment has been minimized.

This comment has been minimized.

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abstratt Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abstratt Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abstratt Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

bot-gradle commented Dec 16, 2025

Uh oh!

abstratt Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abstratt commented Dec 16, 2025

Uh oh!

This comment has been minimized.

bot-gradle commented Dec 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

abstratt commented Dec 16, 2025 •

edited

Loading

abstratt Dec 16, 2025 •

edited

Loading

abstratt Dec 17, 2025 •

edited

Loading

abstratt Dec 16, 2025 •

edited

Loading

abstratt Dec 16, 2025 •

edited

Loading