Skip to content

Conversation

melix
Copy link
Contributor

@melix melix commented Oct 16, 2020

This commit introduces a new DSL, available on Settings via the
dependency resolution management block, which allows declaring aliases
for dependencies and bundles of dependencies.

dependencyResolutionManagement {
    dependenciesModel {
        alias("groovy", "org.codehaus.groovy", "groovy") {
            strictly("3.0.2")
        }
    }
}

Doing this, Gradle will automatically generate type-safe accessors
which are exposed as extensions to all projects. This effectively
allows sharing dependency declarations between projects.

Said differently, this is an officially supported pattern for the
various "dependency version sharing" patterns that we found in the
wild:

  • declaring dependency versions in gradle.properties, then referencing
    the version via project.someLibraryVersion
  • creating a new class in buildSrc which contains dependency coordinates
    and then can be accessed in the different projects
  • creating extra properties for dependency versions at the root project
  • using external plugins like the "refresh dependencies" plugin
  • ...

In addition to this DSL, we introduce a TOML file, which, if found under
the gradle directory, will be used to source dependency aliases.

For example, giving the following file:

[dependencies]
guava = { group = "com.google.guava", name = "guava", version = "27.0-jre" }
groovy = { group = "org.codehaus.groovy", name = "groovy", version.strictly = "3.0.2" }

a dependency can be added in a project using the type-safe libs extension:

dependencies {
    api(libs.guava)
}

This file also supports bundles of dependencies, in case more than one
dependency needs to be added:

[bundles]
groovy = ["groovy", "groovy-json"]

then dependencies can be added via:

dependencies {
   implementation(libs.groovyBundle)
}

The file is configuration-cache safe: any change to the file will invalidate the
cache. But more importantly, it's build cache safe: if the aliases don't change,
there's no need to rebuild the dependent scripts. This means that changing this
file by changing, for example, the dependency versions, will not trigger a
recompilation of build scripts like some of the approaches described above.

The TOML file also lets you declare plugin versions:

[plugins]
my.awesome.plugin="1.0.2"

which allows application of the plugin without version in build scripts:

plugins {
   id 'my.awesome.plugin'
}

In addition, we also generate type-safe accessors for projects. For example:

dependencies {
    api(project(":commons:core"))
}

can be replaced with:

dependencies {
    api(projects.commons.core)
}

Therefore any change to project coordinates, or removals of subprojects would
trigger a build script compilation error, avoiding tedious search and replace.

Fixes #?

Context

Contributor Checklist

Gradle Core Team Checklist

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation
  • Recognize contributor in release notes

@melix melix added this to the 6.8 RC1 milestone Oct 16, 2020
@melix melix self-assigned this Oct 16, 2020
@melix melix force-pushed the cc/dm/central-dependencies branch 3 times, most recently from 60b13a1 to a4f434a Compare October 17, 2020 10:20
@vlsi
Copy link
Contributor

vlsi commented Oct 17, 2020

In addition, we also generate type-safe accessors for projects. For example:

That is awesome.

In addition to this DSL, we introduce a TOML file, which, if found under the gradle directory

Do you think it could support overriding the version with a command-line flag?

What is the approach for file splitting?
What is the approach for plugin versions?

@melix
Copy link
Contributor Author

melix commented Oct 17, 2020

Do you think it could support overriding the version with a command-line flag?

Technically speaking we can. Now I'd have to talk to the larger team to say if we should :) Note that if you override a dependency version, it would probably only override the default model, so if you have something like:

dependencies {
    implementation(libs.guava) {
        version {
           // this wins over whatever is overridden
        }
    }
}

What is the approach for file splitting?

For now I'd say KISS. We could provide alternatives if there's a legitimate interest in making this more complicated. Alternatively settings plugins can provide this feature. Last but not least, because we do code generation we could hit the limit of the number of methods in a class...

What is the approach for plugin versions?

This PR lets you declare this:

[plugins]
my.awesome.plugin="1.4"

then you don't have to specify a version when applying in build scripts.

@vlsi
Copy link
Contributor

vlsi commented Oct 17, 2020

Technically speaking we can. Now I'd have to talk to the larger team to say if we should :)

Let me clarify the use-case.

I want to verify if my project works with a newer version of a third-party dependency.
For instance, I might need to verify if a newer Checkstyle version works for my project, or a newer Guava works for me.
The important point is I don't want to modify the source files because, well, editing the source files is prone to errors.

Note: I can't use composite builds for that, because:

  1. Third-party is not always Gradle based (yet :'( )
  2. composite build differs from the case when dependency is fetched from Maven repository. For instance, composite build ignores Gradle Metatada, so the only way to verify if Gradle Metatada is OK is to install it to a repository (e.g. ~/.m2) and build against it.

Here's the case where I install Apache Calcite Avatica to the local maven repository and build Apache Calcite with that dependency: https://github.com/apache/calcite/blob/cd922deff8ae3f25546b1d77fb147c3098eb177b/.github/workflows/main.yml#L88-L103
I use a hand-crafted version like 1.0.0-dev-master-SNAPSHOT to ensure it does not accidentally fetch an old snapshot from somewhere.

Here's the case when Checkstyle CI job verifies the new Checkstyle works for verifying of pgjdbc sources: https://github.com/checkstyle/checkstyle/blob/3c2249b239cbfe002af3649f1f7c1f9ae61df3b1/.ci/wercker.sh#L60-L71

@vlsi
Copy link
Contributor

vlsi commented Oct 17, 2020

Alternatively settings plugins can provide this feature

I wonder if TOML could be separate from the code generator, so out-of-core plugins could leverage the same generator.

@melix
Copy link
Contributor Author

melix commented Oct 17, 2020

I wonder if TOML could be separate from the code generator, so out-of-core plugins could leverage the same generator.

That's exactly the case: the TOML file is just, if you will, a standard plugin hooking into the code generator. In the end, there's a single code generator and settings plugins can contribute entries. But in the end we're going to be limited by the number of methods which can be defined in a class file.

@melix
Copy link
Contributor Author

melix commented Oct 17, 2020

Let me clarify the use-case.

That is certainly a legitimate use case.

@martinbonnin
Copy link
Contributor

martinbonnin commented Oct 17, 2020

That looks seriously awesome 🤩 . Thanks for addressing this !!

Will the TOML file support dependencies "groups" that need to be all the same version so that it can be changed in a single place? For an example (that obviously won't work since I don't think TOML supports variables) but just to demonstrate the goal:

[versions]
sqldelight = "1.7.0"

[dependencies]
sqlDelightPlugin = { group = "com.squareup.sqldelight", name = "gradle-plugin", version = "${versions.sqldelight}" }
sqlDelightDriverAndroid = { group = "com.squareup.sqldelight", name = "android-driver", version.strictly = "${versions.sqldelight}" }
sqlDelightDriverAndroid = { group = "com.squareup.sqldelight", name = "native-driver", version.strictly = "${versions.sqldelight}" }

Also would it be possible to copy/paste the maven coordinates in one go? That's a minor change but not having to separate group/artifactId/version would be a nice addition as most of the documentations use that representation:

[dependencies]
guava = { gav = "com.google.guava:guava:27.0-jre" }

Which leads me to a wider question: did you consider a Kotlin script for this that would expose a strongly typed way to register dependencies and could be self documented? Something like dependencies.gradle.kts:

dependencies {
  create("guava") {
    group = "com.google.guava"
    artifact = "guava"
    version= "27.0-jre"
  }
  // or the shorthand gav version
  create("guava") {
    gav = "com.google.guava:guava:27.0-jre"
  }
}

bundles {
  create("groovy") {
    from("groovy", "groovy-json")
  }
}

Could that work?

@melix
Copy link
Contributor Author

melix commented Oct 17, 2020

Will the TOML file support dependencies "groups" that need to be all the same version so that it can be changed in a single place?

Not out of the box. If we do this I'd rather add a [versions] section and have the other modules reference it via a versionRef="someId" rather than inventing a substitution mechanism.

Also would it be possible to copy/paste the maven coordinates in one go? That's a minor change but not having to separate group/artifactId/version would be a nice addition as most of the documentations use that representation:

That's already supported by this PR :)

Which leads me to a wider question: did you consider a Kotlin script for this that would expose a strongly typed way to register dependencies and could be self documented?

There's already the settings DSL for this. Note, however, that using Kotlin or the settings DSL would invalidate all build scripts classpath and trigger recompilation of all scripts if any version or coordinate changes, whereas using the TOML this would only happen if you add/remove an alias, not if you change GAV coordinates.

@martinbonnin
Copy link
Contributor