Clean up subproject grouping in gradle build #12805

jjohannes · 2020-04-14T12:26:25Z

This commit improves the grouping of subprojects in the Gradle build by basically getting rid of two things:

A quite messy list of lists of projects for different purposes - ProjectGroups.kt
A ModuleType property each project had to set

Instead, each project now applies one uber plugin indicating it's purpose:

gradlebuild.distribution - part of the distribution. Subcategories determine if the project is in the core of the distribution or if it is a packaged plugin; if it is part of the api or only an implementation detail; if it is written in Java or Kotlin
- gradlebuild.distribution.core-implementation-java
- gradlebuild.distribution.core-implementation-kotlin
- gradlebuild.distribution.core-api-java
- gradlebuild.distribution.core-api-kotlin
- gradlebuild.distribution.plugins-api-java
- gradlebuild.distribution.plugins-implementation-java
- gradlebuild.distribution.plugins-implementation-kotlin
gradlebuild.portalplugin - subproject is not part of distribution but separately published to the plugin portal.
- gradlebuild.portalplugin.kotlin
gradlebuild.internal - internal project
- gradlebuild.internal.java
- gradlebuild.internal.kotlin
- gradlebuild.internal.kotlin-js

Other concerns that cross-cut these categories are realized by additional plugins (e.g. subproject is also published separately) or as a configuration property of the UnitTestAndCompilePlugin extension (e.g. is used in workers). This can be further refined in the future.

This change also removes some configuration time ordering issues.

Following this improvement, I want to get rid of the "grouping" that is done by creating a bunch of configurations in the root build which is then used to assemble the distribution. This should all move into the "distribution" project and the "IntTestImage" plugin. Which both can be simplified now that we have the cleaner structuring that comes with this improvement.

jjohannes · 2020-04-14T12:28:54Z

...ality/src/main/kotlin/org/gradle/gradlebuild/buildquality/DependencyVulnerabilitiesPlugin.kt

 import org.owasp.dependencycheck.gradle.extension.DependencyCheckExtension


 open class DependencyVulnerabilitiesPlugin : Plugin<Project> {


Can we remove this completely? It is not used, out of date (selected plugin version of 'org.owasp.dependencycheck' does not work with current wrapper anymore) and not included in any CI pipeline.

jjohannes · 2020-04-14T12:29:46Z

buildSrc/subprojects/configuration/src/main/kotlin/build-extensions.kt

 @Suppress("unchecked_cast")
 val Project.libraries
-    get() = rootProject.extra["libraries"] as Map<String, Map<String, String>>
+    get() = if (rootProject.extra.has("libraries")) rootProject.extra["libraries"] as Map<String, Map<String, String>> else mapOf()


Need this has check for pre-compiled script plugin compilation. (In a next step we shall get rid of using extra altogether.)

jjohannes · 2020-04-14T12:32:04Z

...tegration-testing/src/main/kotlin/org/gradle/gradlebuild/test/fixtures/IntTestImagePlugin.kt

-            gradleRuntimeSource(project(":apiMetadata"))
-            gradleDocumentation(project(":docs"))
-            gradleScripts(project(":launcher"))
+            gradleRuntimeSource.withDependencies {


This config code is now also executed as part of pre-compiled script compilation. There the projects we try to find here (e.g. project(":apiMetadata")) do not exist. We now only apply the dependencies once the configuration is resolved (withDependencies hook).

jjohannes · 2020-04-14T12:32:59Z

buildSrc/subprojects/kotlin-dsl/src/main/kotlin/plugins/kotlin-dsl-plugin-bundle.gradle.kts


-
-tasks {
-    validatePlugins {


This was a duplication. This is configured for all "java/kotlin" projects already by UnitTestAndCompilePlugin.

jjohannes · 2020-04-14T12:36:18Z

buildSrc/subprojects/plugins/src/main/kotlin/org/gradle/plugins/lifecycle/LifecyclePlugin.kt

    }

+    private
+    fun Project.sharedDependencyAndQualityConfigs() {


I moved this here to make sure this is done as the first thing and is available for all other plugins/scripts afterwards. Then we do not have to be careful of order in the root build script (apart from applying the lifecycle plugin first).

A follow up should be to get rid of storing dependencies in "extra" altogether.

jjohannes · 2020-04-14T12:40:11Z

subprojects/build-scan-performance/build-scan-performance.gradle.kts

-    moduleType = ModuleType.INTERNAL
+// so that all Gradle features are available
+configurations.performanceTestRuntimeOnly.get().withDependencies {
+    addAll(rootProject.configurations.testRuntime.get().allDependencies)


I made this change in some places to get rid of the "extra" properties set in the root build script.

In the next step I want to clean up the distribution building and improve this part further so that we do not need to reach into the root project anymore.

jjohannes · 2020-04-14T16:46:25Z

Full ready-for-release build, before rebase: https://builds.gradle.org/buildConfiguration/Gradle_Check_Stage_ReadyforRelease_Trigger/33575502

jjohannes · 2020-04-15T14:24:10Z

@bot-gradle test this

bot-gradle · 2020-04-15T14:24:20Z

OK, I've already triggered ReadyForMerge build for you.

...ion-testing/src/main/kotlin/org/gradle/gradlebuild/test/fixtures/GradleDistributionPlugin.kt

- Get rid of usages of "extra" - Use Kotlin DSL APIs - Adress or remove outdated TODOs

install task now lives in 'distributions'

Removed this by mistake as it had a misleading comment attached.

ljacomet

Nice clean, thanks for taking the time! 👍

jjohannes self-assigned this Apr 14, 2020

jjohannes added @jvm in:building-gradle gradle/gradle build labels Apr 14, 2020

jjohannes commented Apr 14, 2020

View reviewed changes

jjohannes requested a review from blindpirate April 14, 2020 12:45

jjohannes force-pushed the jjohannes/gradle-build/grouping branch from 319a0b9 to e6cfe48 Compare April 14, 2020 16:46

jjohannes force-pushed the jjohannes/gradle-build/grouping branch from e6cfe48 to 9f562dc Compare April 15, 2020 14:24

gradle deleted a comment from bot-gradle Apr 15, 2020

jjohannes force-pushed the jjohannes/gradle-build/grouping branch from c729b19 to 20cb240 Compare April 16, 2020 11:04

bamboo reviewed Apr 16, 2020

View reviewed changes

...ion-testing/src/main/kotlin/org/gradle/gradlebuild/test/fixtures/GradleDistributionPlugin.kt Outdated Show resolved Hide resolved

jjohannes force-pushed the jjohannes/gradle-build/grouping branch 4 times, most recently from 4350d3f to 89275e4 Compare April 17, 2020 11:10

jjohannes added 7 commits April 17, 2020 13:44

Clean up subproject grouping in gradle build

3f38397

Remove unused build script code

70cc3da

Convert 'distributions' build script to Kotlin DSL

501c832

Move projects to right groups

0b2828d

Cleanup build script code

65cbe91

- Get rid of usages of "extra" - Use Kotlin DSL APIs - Adress or remove outdated TODOs

Remove unused vulnerability check plugin

197e088

Fix and rename 'int test image' plugin to 'distribution plugin'

077b1fe

jjohannes added 6 commits April 17, 2020 13:44

Move more things out of root script and document plugins applied in root

242eb9b

Update subprojects list

81172fb

Fix location of incubating API report

d4ce23a

Fix Gradleception build setup

15275eb

install task now lives in 'distributions'

Use better APIs

ed93e71

Configure repositories without cross-project configurations

9b0de09

jjohannes force-pushed the jjohannes/gradle-build/grouping branch from 89275e4 to 51857e4 Compare April 17, 2020 11:45

jjohannes added 9 commits April 17, 2020 15:37

Add todo for places where cross-project configuration could be removed

e51cfc4

Code cleanup

a4def90

Apply quality config without cross-project configuration

48d3083

Make use of shared fullGradleRuntimeClasspath in arch test project

8f18206

fix formatting

0965485

Update path to install task

8d005aa

Re-add test runtime classpath normalization

f3e0adf

Removed this by mistake as it had a misleading comment attached.

Fixed order for entires in classpath manifest files

70d5e8f

Use jdk11 instead of jdk9 in tests

c78e8bf

jjohannes force-pushed the jjohannes/gradle-build/grouping branch from 51857e4 to 1416e7b Compare April 17, 2020 13:38

ljacomet approved these changes Apr 17, 2020

View reviewed changes

Fix position of 'afterEvaluate' block

32aa2d7

jjohannes force-pushed the jjohannes/gradle-build/grouping branch from 1416e7b to 32aa2d7 Compare April 17, 2020 15:06

jjohannes removed the request for review from blindpirate April 17, 2020 15:06

jjohannes merged commit a69b556 into master Apr 17, 2020

jjohannes deleted the jjohannes/gradle-build/grouping branch May 4, 2020 09:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Clean up subproject grouping in gradle build #12805

Clean up subproject grouping in gradle build #12805

Uh oh!

jjohannes commented Apr 14, 2020 •

edited

Loading

Uh oh!

jjohannes Apr 14, 2020

Uh oh!

jjohannes Apr 14, 2020 •

edited

Loading

Uh oh!

jjohannes Apr 14, 2020

Uh oh!

jjohannes Apr 14, 2020 •

edited

Loading

Uh oh!

jjohannes Apr 14, 2020

Uh oh!

jjohannes Apr 14, 2020

Uh oh!

jjohannes commented Apr 14, 2020 •

edited

Loading

Uh oh!

jjohannes commented Apr 15, 2020

Uh oh!

bot-gradle commented Apr 15, 2020

Uh oh!

Uh oh!

ljacomet left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		import org.owasp.dependencycheck.gradle.extension.DependencyCheckExtension


		open class DependencyVulnerabilitiesPlugin : Plugin<Project> {



		tasks {
		validatePlugins {

Clean up subproject grouping in gradle build #12805

Clean up subproject grouping in gradle build #12805

Uh oh!

Conversation

jjohannes commented Apr 14, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jjohannes Apr 14, 2020

Choose a reason for hiding this comment

Uh oh!

jjohannes Apr 14, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jjohannes Apr 14, 2020

Choose a reason for hiding this comment

Uh oh!

jjohannes Apr 14, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jjohannes Apr 14, 2020

Choose a reason for hiding this comment

Uh oh!

jjohannes Apr 14, 2020

Choose a reason for hiding this comment

Uh oh!

jjohannes commented Apr 14, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jjohannes commented Apr 15, 2020

Uh oh!

bot-gradle commented Apr 15, 2020

Uh oh!

Uh oh!

ljacomet left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jjohannes commented Apr 14, 2020 •

edited

Loading

jjohannes Apr 14, 2020 •

edited

Loading

jjohannes Apr 14, 2020 •

edited

Loading

jjohannes commented Apr 14, 2020 •

edited

Loading