Working with files docs #4293

pledbrook · 2018-02-07T15:31:37Z

DO NOT MERGE!

This is a series of changes to the Working With Files chapter of the user guide that I'm collecting into a pull request to make reviewing easier. Hopefully the changes will eventually propagate to the user guide, but for now they should not be merged.

I have started the chapter with a series of common tasks that users want to accomplish with files. The aim is to answer some commonly asked questions while directing readers to more detailed information so that they have a broader understanding of the topic.

bmuschko

Hi Peter,

I ran over the preliminary content and left a couple of comments. Overall, the content is already much better than before. I am going to give a more thorough pass when you think the changes are in a somewhat final state.

bmuschko · 2018-02-07T18:52:30Z

subprojects/docs/src/docs/userguide/workingWithFiles.adoc

 == Working With Files

-Most builds work with files. Gradle adds some concepts and APIs to help you achieve this.
+Almost every Gradle build works with files: think source files, library dependencies, reports and so on. That's why Gradle comes with a comprehensive API that makes it simple to achieve common tasks.


Instead of "library dependencies" we settled on the term "file dependency". See https://docs.gradle.org/nightly/userguide/declaring_dependencies.html#sec:declaring_file_dependency for example.

Use "file dependency" in place of "library dependency"

bmuschko · 2018-02-07T18:53:03Z

subprojects/docs/src/docs/userguide/workingWithFiles.adoc

 == Working With Files

-Most builds work with files. Gradle adds some concepts and APIs to help you achieve this.
+Almost every Gradle build works with files: think source files, library dependencies, reports and so on. That's why Gradle comes with a comprehensive API that makes it simple to achieve common tasks.


In this section we can probably be more concrete. I'd say "file operations" instead of "common tasks".

Be more concrete and less ambiguous than "common tasks"

bmuschko · 2018-02-07T18:55:51Z

subprojects/docs/src/docs/userguide/workingWithFiles.adoc

+
+You copy a file by creating an instance of Gradle's builtin api:org.gradle.api.tasks.Copy[] task and configuring it with the location of the file and where you want to put it. This example mimics copying a generated report into a directory that will be packed into an archive (zip, tarball, or whatever):
+
+----


All code samples should be an embedded code snippet from an actual build script under docs/src/samples. Furthermore, all of them should be tested e.g. underneath integ-test/src/integTest/groovy/org/gradle/integtests/samples.

Understood. I simply wanted to get the text done first. BTW, are there any instructions for running the docs integration tests or a subset of them? I remember working it out last year, but have of course forgotten and can't find any info on it.

Turn code examples into tested samples

Update existing samples to use best practice and reflect their corresponding points as succinctly and well as possible

There are two types of tests you can run:

The UserGuideSamplesIntegrationTest which basically only verifies that the expected output has been produced according to the *.out that follow the naming convention of the sample ID. Instructions are in the comment of the test class.

The corresponding integration test(s) that execute the sample code and make assertions about the outcome. One example is SamplesDeclaringDependenciesIntegrationTest. After extracting the samples (as described in 1) you can just run it from the IDE. It's possible that they do not exist for this page. I'd set up a new test class if it doesn't exist yet.

When reworking content in the dependency management chapter I have been leaning mostly toward 2 as it provides a way verify more than just the output.

I'm running into an issue trying to execute the dedicated integration tests from IntelliJ:

java.lang.ExceptionInInitializerError at org.gradle.integtests.fixtures.AbstractIntegrationSpec.$spock_initializeFields(AbstractIntegrationSpec.groovy:56) Caused by: org.gradle.internal.service.ServiceCreationException: Could not create service of type ClassLoaderRegistry using GlobalScopeServices.createClassLoaderRegistry(). at org.gradle.integtests.fixtures.executer.AbstractGradleExecuter.<clinit>(AbstractGradleExecuter.java:92) ... 1 more Caused by: java.lang.IllegalArgumentException: Cannot find JAR 'slf4j-api-1.7.16.jar' required by module 'gradle-core' using classpath. at org.gradle.api.internal.classpath.DefaultModuleRegistry.findDependencyJar(DefaultModuleRegistry.java:254) ... 9 more

I imported the the project after running ./gradlew idea. Not sure what else I need to do. Any ideas?

Hhhm, doesn't sounds familiar. Can you try the following?

Run ./gradlew cleanIdea idea from the command line. Do not import the project by pointing to the root build.gradle file. There're still some things that don't work properly yet with the built-in Gradle import functionality. Just open the generated gradle.ipr file.

Run ./gradlew intTestImage from the command line.

Run the test from the IDE.

Thanks. I ran into another issue with a conflicting groovy-all library, but hacked that to get the tests running from within the IDE.

OK, great. Saw your question on the channel. Looks like it was a bug.

bmuschko · 2018-02-07T18:57:14Z

subprojects/docs/src/docs/userguide/workingWithFiles.adoc

+
+We look at these parts in more detail later in the chapter, but we'll start with how you accomplish some of the most common tasks users need.
+
+=== Copying a single file


I think it would make sense to also explain the from/into methods to a certain extent.

Agreed, although I'll keep it minimal. Now that I have the later sections in place, it will be easier to reference them appropriately.

Explain what from() and into() do/are for

bmuschko · 2018-02-07T18:58:58Z

subprojects/docs/src/docs/userguide/workingWithFiles.adoc

+Note that hardcoding paths like this makes the build brittle. It's better to use the task that generates a file as the source of the file's location. In this modified example, we use a report task defined elsewhere that has the report's location stored in the `outputFile` property:
+
+----
+ext.archiveDir = file("${buildDir}/toArchive")


I'd define the extra property for the task and not the project. That will provide a narrower context and can only be used with the reference of this specific task. Ultimately, it avoids introducing global properties which makes the build harder to maintain.

Agreed. I was thinking in terms of a generic directory where all archives are put, but that's not the case here. It's specific to this task.

Move project property onto the task

bmuschko · 2018-02-07T19:03:27Z

subprojects/docs/src/docs/userguide/workingWithFiles.adoc

+
+----
+task packageDistribution(type: Zip) {
+    archiveName = "my-distribution.zip"


Maybe we should promote the use of the base plugin here instead as it provides the conventions.

Since there are two examples, I'd like to have one without the plugin, to demonstrate that it's not needed, and one with to show the convenience. I'll probably trim this section a bit too, as I've now updated the in-depth section on creating archives which covers the base plugin and naming conventions in detail.

Use one sample without the Base Plugin, and one with

bmuschko · 2018-02-07T19:04:59Z

subprojects/docs/src/docs/userguide/workingWithFiles.adoc

+}
+----
+
+=== Moving files and directories


Good to spell it out!

bmuschko · 2018-02-07T19:07:29Z

subprojects/docs/src/docs/userguide/workingWithFiles.adoc

+}
+----
+
+As you can see, you can have multiple `from()` declarations in a copy specification, each with its own configuration.


We should have a section that explain the file operation methods on Project (e.g. project.copy) and the difference between the task types and the methods.

That was already in the chapter, but I've updated it. You'll find it in the File Copying in Depth section. It's actually focused on the copy() method, but now you mention it, there isn't anything on delete() or its corresponding task right now. Are there any others I should cover?

Add information on the following:

Delete task and method

sync() method

I can only think of Project.mkdir() and Project.delete() right now. I don't think we need to show examples for all of them. It's probably OK to just list them and link to the DSL guide.

bmuschko · 2018-02-07T19:08:35Z

subprojects/docs/src/docs/userguide/workingWithFiles.adoc

+
+One thing to note is how the `include()` directive applies only to the `from()`, whereas the directive in the previous section applied to the whole task. These different levels of granularity in the copy specification allow you to easily handle most requirements that you will come across. TODO: Link to more details on copy specs.
+
+=== Creating archives (zip, tar, etc.)


Frequently asked about on the user forum: How do I create an uberjar? It would be helpful to explain it here.

Good idea. If it's that frequently asked, it might make sense to include it as one of the early examples in the chapter, with perhaps some more detail in the in-depth section.

Provide an example on creating an uber JAR

@bmuschko Do you mean one in which the contents of the other JARs are included in the fat JAR, or the JARs themselves are included as is in the fat JAR (the Spring Boot approach — or at least it was).

I meant 1. For example:

task uberjar(type: Jar) { from zipTree(jar1) from zipTree(jar2) }

2 is similar except that the contents of the JAR files (to be included) are not extracted via zipTree.

bmuschko · 2018-02-07T19:15:08Z

subprojects/docs/src/docs/userguide/workingWithFiles.adoc

+
+This is not a common requirement and should be used sparingly as you lose information and can easily break a build. It's generally preferable to copy directories and files instead.
+
+=== Renaming files on copy


One example on more complex processing would be great as well e.g. the use of eachFile.

Is there a common scenario for users in which something like eachFile() is the best solution? Or is it only for specialised cases?

pledbrook · 2018-02-15T09:09:02Z

@bmuschko @eriwen Are we using the guides style guide on the user manual? Is there another document? I hate not knowing how to refer to plugins by name, whether to capitalize archive format names, and so on.

bmuschko · 2018-02-16T22:29:28Z

Are we using the guides style guide on the user manual? Is there another document? I hate not knowing how to refer to plugins by name, whether to capitalize archive format names, and so on.

The guides style guide contains some good advice. I don't think we have been following it in the past when writing user guide content. It would make sense to enhance the style guide by additional rules and then use it across the board. @eriwen Any additional thoughts?

bmuschko · 2018-02-16T22:50:31Z

subprojects/docs/docs.gradle


-    attributes website: 'http://www.gradle.org'
+    attributes website: 'http://www.gradle.org',
+               javaApi: 'https://docs.oracle.com/javase/8/docs/api'


We might want to reference the Java 7 docs here as we still supporting it as the minimal JDK version.

I'm fine with that. One of the reasons I made it a document attribute :)

Reference Java 7 API docs rather than Java 8

pledbrook added 4 commits February 2, 2018 13:59

Restructure section on file collections et al.

caf531d

Merge branch 'master' into working-with-files-docs

4c30a1a

Add missing image files

fe18c54

eriwen self-requested a review February 7, 2018 15:53

eriwen assigned pledbrook Feb 7, 2018

eriwen added from:member in:documentation-infrastructure labels Feb 7, 2018

eriwen requested a review from bmuschko February 7, 2018 15:55

bmuschko suggested changes Feb 7, 2018

View reviewed changes

Finish first draft of Working with Files update

20ceae0

Incorporate some of bmuschko's feedback

337ca34

bmuschko reviewed Feb 16, 2018

View reviewed changes


		You copy a file by creating an instance of Gradle's builtin api:org.gradle.api.tasks.Copy[] task and configuring it with the location of the file and where you want to put it. This example mimics copying a generated report into a directory that will be packed into an archive (zip, tarball, or whatever):

		----


		We look at these parts in more detail later in the chapter, but we'll start with how you accomplish some of the most common tasks users need.

		=== Copying a single file


		One thing to note is how the `include()` directive applies only to the `from()`, whereas the directive in the previous section applied to the whole task. These different levels of granularity in the copy specification allow you to easily handle most requirements that you will come across. TODO: Link to more details on copy specs.

		=== Creating archives (zip, tar, etc.)


		This is not a common requirement and should be used sparingly as you lose information and can easily break a build. It's generally preferable to copy directories and files instead.

		=== Renaming files on copy

Working with files docs #4293

Working with files docs #4293

Uh oh!

Conversation

pledbrook commented Feb 7, 2018

Uh oh!

bmuschko left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pledbrook Feb 16, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bmuschko Feb 16, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pledbrook Feb 16, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pledbrook Feb 16, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pledbrook Feb 16, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bmuschko Feb 16, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

pledbrook Feb 16, 2018 •

edited

Loading

bmuschko Feb 16, 2018 •

edited

Loading

pledbrook Feb 16, 2018 •

edited

Loading

pledbrook Feb 16, 2018 •

edited

Loading

pledbrook Feb 16, 2018 •

edited

Loading

bmuschko Feb 16, 2018 •

edited

Loading

pledbrook Feb 21, 2018 •

edited

Loading