software.amazon.awssdk.transfer.s3

Interface S3TransferManager

All Superinterfaces:

AutoCloseable, SdkAutoCloseable

All Known Implementing Classes:

DefaultS3TransferManager

public interface S3TransferManager
extends SdkAutoCloseable

The S3 Transfer Manager is a library that allows users to easily and optimally upload and downloads to and from S3. Usage Example:

 
 // Create using all default configuration values
 S3TransferManager tm = S3TransferManager.create();

 // If you wish to configure settings, we recommend using the builder instead
 S3TransferManager tm =
     S3TransferManager.builder()
                      .s3ClientConfiguration(b -> b.credentialsProvider(credentialProvider)
                                                   .region(Region.US_WEST_2)
                                                   .targetThroughputInGbps(20.0)
                                                   .minimumPartSizeInBytes(8 * MB))
                      .build();

 // Upload a file to S3
 FileUpload upload =
     tm.uploadFile(u -> u.source(Paths.get("myFile.txt"))
                         .putObjectRequest(p -> p.bucket("bucket").key("key")));
 upload.completionFuture().join();

 // Download an S3 object to a file
 FileDownload download =
     tm.downloadFile(d -> d.getObjectRequest(g -> g.bucket("bucket").key("key"))
                           .destination(Paths.get("myFile.txt")));
 download.completionFuture().join();
 
 // Upload any content to S3
 Upload upload =
     tm.upload(u -> u.requestBody(AsyncRequestBody.fromString("Hello world"))
                     .putObjectRequest(p -> p.bucket("bucket").key("key")));
 upload.completionFuture().join();
 
 // Download an S3 object to a custom destination
 Download<ResponseBytes<GetObjectResponse>> download =
     tm.download(d -> d.getObjectRequest(g -> g.bucket("bucket").key("key"))
                       .responseTransformer(AsyncResponseTransformer.toBytes()));
 download.completionFuture().join();
 
 // Attach a TransferListener
 FileUpload upload =
     tm.uploadFile(u -> u.source(Paths.get("myFile.txt"))
                         .putObjectRequest(p -> p.bucket("bucket").key("key"))
                         .overrideConfiguration(o -> o.addListener(LoggingTransferListener.create())));
 upload.completionFuture().join();
 
 

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static interface	S3TransferManager.Builder The builder definition for a S3TransferManager.

Method Summary

Modifier and Type	Method and Description
static S3TransferManager.Builder	builder() Creates a default builder for S3TransferManager.
static S3TransferManager	create() Create an S3TransferManager using the default values.
default <ResultT> Download<ResultT>	download(DownloadRequest<ResultT> downloadRequest) Download an object identified by the bucket and key from S3 through the given AsyncResponseTransformer.
default <ResultT> Download<ResultT>	download(Function<DownloadRequest.UntypedBuilder,DownloadRequest.TypedBuilder<ResultT>> request) This is a convenience method that creates an instance of the DownloadRequest builder, avoiding the need to create one manually via DownloadRequest.builder().
default FileDownload	downloadFile(Consumer<DownloadFileRequest.Builder> request) This is a convenience method that creates an instance of the DownloadFileRequest builder, avoiding the need to create one manually via DownloadFileRequest.builder().
default FileDownload	downloadFile(DownloadFileRequest downloadRequest) Download an object identified by the bucket and key from S3 to a local file.
default Upload	upload(Consumer<UploadRequest.Builder> request) This is a convenience method that creates an instance of the UploadRequest builder, avoiding the need to create one manually via UploadRequest.builder().
default Upload	upload(UploadRequest uploadRequest) Upload the given AsyncRequestBody to an object in S3.
default DirectoryUpload	uploadDirectory(Consumer<UploadDirectoryRequest.Builder> requestBuilder) This is a convenience method that creates an instance of the UploadDirectoryRequest builder, avoiding the need to create one manually via UploadDirectoryRequest.builder().
default DirectoryUpload	uploadDirectory(UploadDirectoryRequest uploadDirectoryRequest) Upload all files under the given directory to the provided S3 bucket.
default FileUpload	uploadFile(Consumer<UploadFileRequest.Builder> request) This is a convenience method that creates an instance of the UploadFileRequest builder, avoiding the need to create one manually via UploadFileRequest.builder().
default FileUpload	uploadFile(UploadFileRequest uploadFileRequest) Upload a local file to an object in S3.

Methods inherited from interface software.amazon.awssdk.utils.SdkAutoCloseable

close

Method Detail

downloadFile

default FileDownload downloadFile(DownloadFileRequest downloadRequest)

Download an object identified by the bucket and key from S3 to a local file. For non-file-based downloads, you may use download(DownloadRequest) instead.

Usage Example:

 
 // Initiate the transfer
 FileDownload download =
     tm.downloadFile(d -> d.getObjectRequest(g -> g.bucket("bucket").key("key"))
                           .destination(Paths.get("myFile.txt")));
 // Wait for the transfer to complete
 download.completionFuture().join();
 
 

See Also:

downloadFile(Consumer), download(DownloadRequest)

downloadFile

default FileDownload downloadFile(Consumer<DownloadFileRequest.Builder> request)

This is a convenience method that creates an instance of the DownloadFileRequest builder, avoiding the need to create one manually via DownloadFileRequest.builder().

See Also:

downloadFile(DownloadFileRequest)

download

default <ResultT> Download<ResultT> download(DownloadRequest<ResultT> downloadRequest)

Download an object identified by the bucket and key from S3 through the given AsyncResponseTransformer. For downloading to a file, you may use downloadFile(DownloadFileRequest) instead.

Usage Example (this example buffers the entire object in memory and is not suitable for large objects):

 
 // Initiate the transfer
 Download<ResponseBytes<GetObjectResponse>> download =
     tm.download(d -> d.getObjectRequest(g -> g.bucket("bucket").key("key"))
                       .responseTransformer(AsyncResponseTransformer.toBytes()));
 // Wait for the transfer to complete
 download.completionFuture().join();
 
 

See the static factory methods available in AsyncResponseTransformer for other use cases.

Type Parameters:

ResultT - The type of data the AsyncResponseTransformer produces

Parameters:

downloadRequest - the download request, containing a GetObjectRequest and AsyncResponseTransformer

Returns:

A Download that can be used to track the ongoing transfer

See Also:

download(Function), downloadFile(DownloadFileRequest)

download

default <ResultT> Download<ResultT> download(Function<DownloadRequest.UntypedBuilder,DownloadRequest.TypedBuilder<ResultT>> request)

This is a convenience method that creates an instance of the DownloadRequest builder, avoiding the need to create one manually via DownloadRequest.builder().

See Also:

download(DownloadRequest)

uploadFile

default FileUpload uploadFile(UploadFileRequest uploadFileRequest)

Upload a local file to an object in S3. For non-file-based uploads, you may use upload(UploadRequest) instead.

Usage Example:

 
 FileUpload upload =
     tm.uploadFile(u -> u.source(Paths.get("myFile.txt"))
                         .putObjectRequest(p -> p.bucket("bucket").key("key")));
 // Wait for the transfer to complete
 upload.completionFuture().join();
 
 

See Also:

uploadFile(Consumer), upload(UploadRequest)

uploadFile

default FileUpload uploadFile(Consumer<UploadFileRequest.Builder> request)

This is a convenience method that creates an instance of the UploadFileRequest builder, avoiding the need to create one manually via UploadFileRequest.builder().

See Also:

uploadFile(UploadFileRequest)

upload

default Upload upload(UploadRequest uploadRequest)

Upload the given AsyncRequestBody to an object in S3. For file-based uploads, you may use uploadFile(UploadFileRequest) instead.

Usage Example:

 
 Upload upload =
     tm.upload(u -> u.requestBody(AsyncRequestBody.fromString("Hello world"))
                     .putObjectRequest(p -> p.bucket("bucket").key("key")));
 // Wait for the transfer to complete
 upload.completionFuture().join();
 
 

See the static factory methods available in AsyncRequestBody for other use cases.

Parameters:

uploadRequest - the upload request, containing a PutObjectRequest and AsyncRequestBody

Returns:

An Upload that can be used to track the ongoing transfer

See Also:

upload(Consumer), uploadFile(UploadFileRequest)

upload

default Upload upload(Consumer<UploadRequest.Builder> request)

This is a convenience method that creates an instance of the UploadRequest builder, avoiding the need to create one manually via UploadRequest.builder().

See Also:

upload(UploadRequest)

uploadDirectory

default DirectoryUpload uploadDirectory(UploadDirectoryRequest uploadDirectoryRequest)

Upload all files under the given directory to the provided S3 bucket. The key name transformation depends on the optional prefix and delimiter provided in the UploadDirectoryRequest. By default, all subdirectories will be uploaded recursively, and symbolic links are not followed automatically. This behavior can be configured in UploadDirectoryOverrideConfiguration at request level via UploadDirectoryRequest.Builder.overrideConfiguration(UploadDirectoryOverrideConfiguration) or client level via S3TransferManager.Builder.transferConfiguration(S3TransferManagerOverrideConfiguration) Note that request-level configuration takes precedence over client-level configuration.

The returned CompletableFuture only completes exceptionally if the request cannot be attempted as a whole (the source directory provided does not exist for example). The future completes successfully for partial successful requests, i.e., there might be failed uploads in the successfully completed response. As a result, you should check for errors in the response via CompletedDirectoryUpload.failedTransfers() even when the future completes successfully.

The current user must have read access to all directories and files

Usage Example:

 
 DirectoryUpload directoryUpload =
       transferManager.uploadDirectory(UploadDirectoryRequest.builder()
                                                             .sourceDirectory(Paths.get("."))
                                                             .bucket("bucket")
                                                             .prefix("prefix")
                                                             .build());
 // Wait for the transfer to complete
 CompletedDirectoryUpload completedDirectoryUpload = directoryUpload.completionFuture().join();

 // Print out the failed uploads
 completedDirectoryUpload.failedTransfers().forEach(System.out::println);

 
 

Parameters:

uploadDirectoryRequest - the upload directory request

See Also:

uploadDirectory(Consumer), UploadDirectoryOverrideConfiguration

uploadDirectory

default DirectoryUpload uploadDirectory(Consumer<UploadDirectoryRequest.Builder> requestBuilder)

This is a convenience method that creates an instance of the UploadDirectoryRequest builder, avoiding the need to create one manually via UploadDirectoryRequest.builder().

See Also:

uploadDirectory(UploadDirectoryRequest)

create

static S3TransferManager create()

Create an S3TransferManager using the default values.

builder

static S3TransferManager.Builder builder()

Creates a default builder for S3TransferManager.