# AWS S3

The AWS S3 connector provides Akka Stream sources and sinks to connect to [Amazon S3](https://aws.amazon.com/s3/).
S3 stands for Simple Storage Service and is an object storage service with a web service interface.

@@project-info{ projectId="s3" }

## Artifacts

@@dependency [sbt,Maven,Gradle] {
  group=com.lightbend.akka
  artifact=akka-stream-alpakka-s3_$scala.binary.version$
  version=$project.version$
}

The table below shows direct dependencies of this module and the second tab shows all libraries it depends on transitively.

@@dependencies { projectId="s3" }

## Configuration

The settings for the S3 connector are read by default from `alpakka.s3` configuration section.
Credentials are loaded as described in the @javadoc[DefaultAWSCredentialsProviderChain](com.amazonaws.auth.DefaultAWSCredentialsProviderChain) documentation.
Therefore, if you are using Alpakka S3 connector in a standard environment, no configuration changes should be necessary.
However, if you use a non-standard configuration path or need multiple different configurations, please refer to @ref[the attributes section below](s3.md#apply-s3-settings-to-a-part-of-the-stream) to see how to apply different configuration to different parts of the stream.
All of the available configuration settings can be found in the @github[reference.conf](/s3/src/main/resources/reference.conf).

## Store a file in S3

A file can be uploaded to S3 by creating a source of @scala[@scaladoc[ByteString](akka.util.ByteString)]@java[@javadoc[ByteString](akka.util.ByteString)] and running that with a sink created from @scala[@scaladoc[S3.multipartUpload](akka.stream.alpakka.s3.scaladsl.S3$)]@java[@scaladoc[S3.multipartUpload](akka.stream.alpakka.s3.javadsl.S3$)].

Scala
: @@snip [snip](/s3/src/test/scala/docs/scaladsl/S3SinkSpec.scala) { #upload }

Java
: @@snip [snip](/s3/src/test/java/docs/javadsl/S3Test.java) { #upload }

## Download a file from S3

A source for downloading a file can be created by calling @scala[@scaladoc[S3.download](akka.stream.alpakka.s3.scaladsl.S3$)]@java[@scaladoc[S3.download](akka.stream.alpakka.s3.javadsl.S3$)].
It will emit an @scala[`Option`]@java[`Optional`] that will hold file's data and metadata or will be empty if no such file can be found.

Scala
: @@snip [snip](/s3/src/test/scala/docs/scaladsl/S3SourceSpec.scala) { #download }

Java
: @@snip [snip](/s3/src/test/java/docs/javadsl/S3Test.java) { #download }

In order to download a range of a file's data you can use overloaded method which
additionally takes `ByteRange` as argument.

Scala
: @@snip [snip](/s3/src/test/scala/docs/scaladsl/S3SourceSpec.scala) { #rangedDownload }

Java
: @@snip [snip](/s3/src/test/java/docs/javadsl/S3Test.java) { #rangedDownload }

File metadata (@scaladoc[ObjectMetadata](akka.stream.alpakka.s3.ObjectMetadata)) holds content type, size and other useful information about the object.
Here's an example of using this metadata to stream an object back to a client in Akka Http.

Scala
: @@snip [snip](/s3/src/test/scala/docs/scaladsl/S3SourceSpec.scala) { #downloadToAkkaHttp }

Java
: @@snip [snip](/s3/src/test/java/docs/javadsl/S3Test.java) { #downloadToAkkaHttp }

## Access object metadata without downloading object from S3

If you do not need object itself, you can query for only object metadata using a source from @scala[@scaladoc[S3.getObjectMetadata](akka.stream.alpakka.s3.scaladsl.S3$)]@java[@scaladoc[S3.getObjectMetadata](akka.stream.alpakka.s3.javadsl.S3$)].

Scala
: @@snip [snip](/s3/src/test/scala/docs/scaladsl/S3SourceSpec.scala) { #objectMetadata }

Java
: @@snip [snip](/s3/src/test/java/docs/javadsl/S3Test.java) { #objectMetadata }

## List bucket contents

To get a list of all objects in a bucket, use @scala[@scaladoc[S3.listBucket](akka.stream.alpakka.s3.scaladsl.S3$)]@java[@scaladoc[S3.listBucket](akka.stream.alpakka.s3.javadsl.S3$)].
When run, this will give a stream of @scaladoc[ListBucketResultContents](akka.stream.alpakka.s3.ListBucketResultContents).

Scala
: @@snip [snip](/s3/src/test/scala/docs/scaladsl/S3SourceSpec.scala) { #list-bucket }

Java
: @@snip [snip](/s3/src/test/java/docs/javadsl/S3Test.java) { #list-bucket }

## Copy upload (multi part)

Copy an S3 object from source bucket to target bucket using @scala[@scaladoc[S3.multipartCopy](akka.stream.alpakka.s3.scaladsl.S3$)]@java[@scaladoc[S3.multipartCopy](akka.stream.alpakka.s3.javadsl.S3$)].
When run, this will emit a single @scaladoc[MultipartUploadResult](akka.stream.alpakka.s3.MultipartUploadResult) with the information about the copied object.

Scala
: @@snip [snip](/s3/src/test/scala/docs/scaladsl/S3SinkSpec.scala) { #multipart-copy }

Java
: @@snip [snip](/s3/src/test/java/docs/javadsl/S3Test.java) { #multipart-copy }

If your bucket has versioning enabled, you could have multiple versions of the same object.
By default AWS identifies the current version of the object to copy.
You can optionally specify a specific version of the source object to copy by adding the `sourceVersionId` parameter.

Scala
: @@snip [snip](/s3/src/test/scala/docs/scaladsl/S3SinkSpec.scala) { #multipart-copy-with-source-version }

Java
: @@snip [snip](/s3/src/test/java/docs/javadsl/S3Test.java) { #multipart-copy-with-source-version }

Different options are available for server side encryption in the @scaladoc[ServerSideEncryption](akka.stream.alpakka.s3.headers.ServerSideEncryption$) fatory.

Scala
: @@snip [snip](/s3/src/test/scala/docs/scaladsl/S3SinkSpec.scala) { #multipart-copy-sse }

Java
: @@snip [snip](/s3/src/test/java/docs/javadsl/S3Test.java) { #multipart-copy-sse }

More S3 specific headers and arbitrary HTTP headers can be specified by adding to the @scaladoc[S3Headers](akka.stream.alpakka.s3.S3Headers) container.

## Apply S3 settings to a part of the stream

It is possible to make one part of the stream use different @scaladoc[S3Settings](akka.stream.alpakka.s3.S3Settings) from the rest of the graph.
This can be useful, when one stream is used to copy files across regions or even different S3 compatible endpoints.
You can attach a custom `S3Settings` instance or a custom config path to a graph using attributes from @scaladoc[S3Attributes](akka.stream.alpakka.s3.S3Attributes$):

Scala
: @@snip [snip](/s3/src/test/scala/docs/scaladsl/S3SourceSpec.scala) { #list-bucket-attributes }

Java
: @@snip [snip](/s3/src/test/java/docs/javadsl/S3Test.java) { #list-bucket-attributes }

## Running the example code

The code in this guide is part of runnable tests of this project. You are welcome to edit the code and run it in sbt.

Scala
:   ```
    sbt
    > s3/test
    ```

Java
:   ```
    sbt
    > s3/test
    ```

> Some test code requires [Minio](https://github.com/minio/minio) running in the background. You can start one quickly using docker:
>
> `docker-compose up minio_prep`
