AWS Developer Tools Blog
AWS SDK for Java 2.x released
We’re pleased to announce that the AWS SDK for Java 2.x is now generally available and supported for production use.
Version 2.x is a major rewrite of the 1.11.x code base. Built with support for Java 8+, 2.x adds several frequently requested features, like nonblocking I/O, improved start-up performance and automatic iteration over paginated responses. In addition, many aspects of the SDK have been updated with a focus on consistency, immutability, and ease of use.
Moving to version 2.x is easy, because it can be run in the same JVM as 1.11.x. This enables you to take advantage of the features you want from 2.x without having to migrate your entire product. Version 2.x includes most of features currently in 1.11.x, but not all. We’re releasing it immediately so you can take advantage of the new features without needing to wait for features you might not use. See the end of this post for the list of 1.11.x features that haven’t yet made it to 2.x.
Although we’re excited about 2.x, we also want to reassure customers that we will continue to update 1.11.x with new service APIs, new services, bug fixes and security fixes.
New features
Several new features have been added in 2.x, including nonblocking I/O, a pluggable HTTP layer, and HTTP client sharing.
Nonblocking I/O
The AWS SDK for Java 2.x utilizes a new, nonblocking SDK architecture built on Netty to support true nonblocking I/O. The 1.11.x version already has asynchronous variants of service clients, but they are a managed thread pool on top of the synchronous clients, so each request still requires its own thread. 2.x asynchronous clients are nonblocking all the way to the HTTP layer, allowing higher concurrency with a small, fixed number of threads.
The asynchronous clients immediately return a CompletableFuture of the response, instead of blocking the thread until the response is available. Exceptions are delivered through the future, instead of being thrown by the client method call.
// Create a default async client with credentials and AWS Region loaded from
// the environment
DynamoDbAsyncClient client = DynamoDbAsyncClient.create();
// Start the call to Amazon Dynamo DB, not blocking to wait for it to finish
CompletableFuture<ListTablesResponse> responseFuture = client.listTables();
// Map the response to another CompletableFuture containing just the table names
CompletableFuture<List<String>> tableNamesFuture =
responseFuture.thenApply(ListTablesResponse::tableNames);
// When future is complete (either successfully or in error), handle the response
CompletableFuture<List<String>> operationCompleteFuture =
tableNamesFuture.whenComplete((tableNames, exception) -> {
if (tableNames != null) {
// Print the table names.
tableNames.forEach(System.out::println);
} else {
// Handle the error.
exception.printStackTrace();
}
});
// We could do other work while waiting for the AWS call to complete in
// the background, but we'll just wait for "whenComplete" to finish instead
operationCompleteFuture.join();Other asynchronous operations that involve streaming input, like the Amazon S3 PutObject, are a little different from their nonstreaming counterparts. They use AsyncRequestBody, an adaptation of the reactive streams interfaces. This ensures that even streaming data to AWS is a nonblocking operation.
// Create a default async client with credentials and AWS Region loaded from the
// environment
S3AsyncClient client = S3AsyncClient.create();
// Start the call to Amazon S3, not blocking to wait for the result
CompletableFuture<PutObjectResponse> responseFuture =
client.putObject(PutObjectRequest.builder()
.bucket("my-bucket")
.key("my-object-key")
.build(),
AsyncRequestBody.fromFile(Paths.get("my-file.in")));
// When future is complete (either successfully or in error), handle the response
CompletableFuture<PutObjectResponse> operationCompleteFuture =
responseFuture.whenComplete((putObjectResponse, exception) -> {
if (putObjectResponse != null) {
// Print the object version
System.out.println(putObjectResponse.versionId());
} else {
// Handle the error
exception.printStackTrace();
}
});
// We could do other work while waiting for the AWS call to complete in
// the background, but we'll just wait for "whenComplete" to finish instead
operationCompleteFuture.join();Asynchronous operations involving streaming output, like the Amazon S3 GetObject, use AsyncResponseTransformer, another adaptation of reactive streams interfaces. This ensures downloading data from AWS is also a nonblocking operation.
// Creates a default async client with credentials and AWS Region loaded from the
// environment
S3AsyncClient client = S3AsyncClient.create();
// Start the call to Amazon S3, not blocking to wait for the result
CompletableFuture<GetObjectResponse> responseFuture =
client.getObject(GetObjectRequest.builder()
.bucket("my-bucket")
.key("my-object-key")
.build(),
AsyncResponseTransformer.toFile(Paths.get("my-file.out")));
// When future is complete (either successfully or in error), handle the response
CompletableFuture<GetObjectResponse> operationCompleteFuture =
responseFuture.whenComplete((getObjectResponse, exception) -> {
if (getObjectResponse != null) {
// At this point, the file my-file.out has been created with the data
// from S3; let's just print the object version
System.out.println(getObjectResponse.versionId());
} else {
// Handle the error
exception.printStackTrace();
}
});
// We could do other work while waiting for the AWS call to complete in
// the background, but we'll just wait for "whenComplete" to finish instead
operationCompleteFuture.join();Automatic pagination
To maximize availability and minimize latency, many AWS APIs break up a result across multiple “pages” of responses. In 1.11.x, customers had to make multiple manual requests in order to access every page of responses. In 2.x, the SDK can handle this automatically, without requiring you to do the heavy lifting.
For example, the following 2.x code prints all of your Dynamo DB table names in the currently-configured Region.
// 2.x pagination
DynamoDbClient client = DynamoDbClient.create();
client.listTablesPaginator()
.tableNames()
.forEach(System.out::println);In 1.11.x, this same behavior requires a large amount of boilerplate code.
// 1.11.x pagination
AmazonDynamoDB client = AmazonDynamoDBClientBuilder.defaultClient();
ListTablesRequest request = new ListTablesRequest();
ListTablesResult result;
do {
result = client.listTables(request);
result.getTableNames()
.forEach(System.out::println);
request.setExclusiveStartTableName(result.getLastEvaluatedTableName());
} while (request.getExclusiveStartTableName() != null);Pluggable HTTP layer
The AWS SDK for Java 1.11.x is tightly coupled to the Apache HTTP client in order to invoke AWS APIs. Although this works well in general, there are often benefits to using a client that is more optimized for your runtime environment. Version 2.x continues to ship Apache as the default synchronous HTTP client, but you can replace it with another implementation that better suits your use-case.
When you first add the SDK to your project and create a client, a default HTTP implementation is selected automatically.
// Create a synchronous client using the Apache HTTP client
DynamoDbClient client = DynamoDbClient.create();
// Create an asynchronous client using the Netty HTTP client
DynamoDbAsyncClient asyncClient = DynamoDbAsyncClient.create();There are some cases where the default isn’t optimal. For example, in AWS Lambda, where startup time is one of the biggest latency concerns, you might want to use an HTTP client based on the JVM’s lightweight URLConnection, instead of Apache’s higher-throughput, but slower-to-start HTTP client.
Version 2.x includes such a URLConnection HTTP client, and it’s simple to use. First, add url-connection-client as a new dependency of your application. If you’re using Maven, this is just a new entry in your pom.xml file.
<!-- pom.xml -->
<project>
...
<dependencies>
...
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>url-connection-client</artifactId>
<version>2.1.0</version>
</dependency>
</dependencies>
...
</project>Then, configure your service in one of the following ways.
Option 1: Specify the HTTP client to use at client creation time.
To use multiple HTTP clients in the same application, you must specify which one you wish to use when you create the client.
// Create a synchronous client using the URLConnection HTTP client
DynamoDbClient clientWithUrlConnectionHttp =
DynamoDbClient.builder()
.httpClientBuilder(UrlConnectionHttpClient.builder())
.build();
// Create a synchronous client using the Apache HTTP client
DynamoDbClient clientWithApacheHttp =
DynamoDbClient.builder()
.httpClientBuilder(ApacheHttpClient.builder())
.build();Option 2: Change the default HTTP client using a system property at JVM startup.
Select the HTTP client at Java startup by using the software.amazon.awssdk.http.service.impl system property for synchronous HTTP clients, or software.amazon.awssdk.http.async.service.impl for asynchronous HTTP clients.
# Specify the default synchronous HTTP client as UrlConnectionHttpClient
java -Dsoftware.amazon.awssdk.http.service.impl=\
software.amazon.awssdk.http.urlconnection.UrlConnectionSdkHttpService \
MyService.jar
# Specify the default synchronous HTTP client as ApacheHttpClient
java -Dsoftware.amazon.awssdk.http.service.impl=\
software.amazon.awssdk.http.apache.ApacheSdkHttpService \
MyService.jar
# Specify the default asynchronous HTTP client as NettyNioAsyncHttpClient
java -Dsoftware.amazon.awssdk.http.async.service.impl=\
software.amazon.awssdk.http.nio.netty.NettySdkAsyncHttpService \
MyService.jarOption 3: Change the default HTTP client using a system property in Java code.
Instead of specifying the system properties listed previously when you start your JVM, you can also specify the system properties at runtime. Be sure to specify the values before you create any clients, or your values won’t be used.
// Set the default synchronous HTTP client to UrlConnectionHttpClient
System.setProperty("software.amazon.awssdk.http.service.impl",
"software.amazon.awssdk.http.urlconnection.UrlConnectionSdkHttpService");
// Set the default synchronous HTTP client to ApacheHttpClient
System.setProperty("software.amazon.awssdk.http.service.impl",
"software.amazon.awssdk.http.apache.ApacheSdkHttpService");
// Set the default asynchronous HTTP client to NettyNioAsyncHttpClient
System.setProperty("software.amazon.awssdk.http.async.service.impl",
"software.amazon.awssdk.http.nio.netty.NettySdkAsyncHttpService");
HTTP client sharing
In 1.11.x of the SDK, each AWS service client instance had its own HTTP client with separate connections and resources. This is still the default in 2.x, but now you can share a single HTTP client among multiple AWS clients. This is useful in resource-constrained environments where you might want to share a single pool of connections among multiple AWS services.
// Create an HTTP client with a maximum of 50 connections to share among
// multiple AWS clients
SdkHttpClient httpClient = ApacheHttpClient.builder()
.maxConnections(50)
.build();
// Create a DynamoDB client using the shared HTTP client
DynamoDbClient dynamoDb = DynamoDbClient.builder()
.httpClient(httpClient)
.build();
// Create an S3 client using the shared HTTP client
S3Client s3Client = S3Client.builder()
.httpClient(httpClient)
.build();Because the HTTP client is no longer tied to a single AWS client, you need to close the HTTP client yourself when you’re done using it.
httpClient.close();
Improvements
In addition to new features, every API in the SDK has been updated to improve consistency, usability, and thread safety. You can browse our full changelog to find everything, but we highlight the biggest improvements below.
Immutability
Many objects are mutable in 1.11.x, which means that their use requires more care to ensure that they aren’t mutated in a multithreaded environment. In 2.x, all objects used by the SDK are immutable and created using builders. This allows you to use the SDK in concurrent environments without worrying about whether other threads might make unsafe mutations.
// Create an immutable SDK configuration object using a configuration builder
ClientOverrideConfiguration overrideConfiguration =
ClientOverrideConfiguration.builder()
.apiCallTimeout(Duration.ofSeconds(30))
.apiCallAttemptTimeout(Duration.ofSeconds(10))
.build();
// Create an immutable DynamoDB client using a client builder
DynamoDbClient dynamoDbClient =
DynamoDbClient.builder()
.overrideConfiguration(overrideConfiguration)
.build();
// Create an immutable DynamoDB ListTablesRequest using a request builder
ListTablesRequest request = ListTablesRequest.builder().build();
// Get an immutable DynamoDB ListTablesResponse using the client
ListTablesResponse response = dynamoDbClient.listTables(request);All objects follow the same pattern: to create it, call its builder() method, configure it, and call build().
In some circumstances, it might be beneficial to modify an immutable object. Although the immutable objects can’t be modified directly, 2.x makes it easy to create a copy of the immutable object with some values modified.
// Create client configuration with timeouts enabled
ClientOverrideConfiguration configurationWithTimeouts =
ClientOverrideConfiguration.builder()
.apiCallTimeout(Duration.ofSeconds(30))
.apiCallAttemptTimeout(Duration.ofSeconds(10))
.build();
// Create a DynamoDB client with timeouts enabled
DynamoDbClient clientWithTimeouts =
DynamoDbClient.builder()
.overrideConfiguration(configurationWithTimeouts)
.build();
// Create a configuration with timeouts and automatic retries disabled
ClientOverrideConfiguration configurationWithTimeoutsAndNoRetries =
configurationWithTimeouts.toBuilder()
.retryPolicy(RetryPolicy.none())
.build();
// Create a DynamoDB client with timeouts and automatic retries disabled
DynamoDbClient clientWithTimeoutsAndNoRetries =
DynamoDbClient.builder()
.overrideConfiguration(configurationWithTimeoutsAndNoRetries)
.build();Although immutability and builders ensure thread safety and provide a consistent pattern for creating objects across 2.x, they add verbosity. For this reason, 2.x exposes optional lambda-style methods that remove much of the ceremony around calling builder() and build().
Choose the one you like most to use in your code base.
Regions
In 1.11.x, multiple classes exist for managing AWS Regions and Region metadata. In 2.x, these have been simplified with the new Region class.
// Create a DynamoDB client that communicates with the us-west-2 (Oregon) Region
DynamoDbClient dynamoDbClient = DynamoDbClient.builder()
.region(Region.US_WEST_2)
.build();
// Get all of the Regions in which DynamoDB is supported
List<Region> dynamoDbRegions = DynamoDbClient.serviceMetadata().regions();
// Get all Regions known to the current SDK version
List<Region> allRegions = Region.regions();
// Specify a newly deployed Region that is unknown to the current SDK version
Region someNewRegion = Region.of("us-west-42");
// Get a human-readable description of this Region (e.g., US West (Oregon))
String regionDescription = Region.US_WEST_2.metadata().description();Streaming API type conversions
In 1.11.x, uploading data to Amazon was done through input streams, with S3 also supporting files and strings. You had to map any other data types to one of these types to send them to Amazon. Version 2.x has been altered to support a large variety of streaming input types, to reduce the effort it takes to communicate with any streaming service.
S3Client s3Client = S3Client.create();
PutObjectRequest request = PutObjectRequest.builder()
.bucket("my-bucket")
.key("my-key")
.build();
s3Client.putObject(request, RequestBody.fromString("MyString"));
s3Client.putObject(request, RequestBody.fromFile(Paths.get("MyFile.in")));
s3Client.putObject(request, RequestBody.fromBytes("MyBytes".getBytes(UTF_8)));
s3Client.putObject(request, RequestBody.fromByteBuffer(ByteBuffer.wrap("MyByteBuffer".getBytes(UTF_8))));
s3Client.putObject(request, RequestBody.fromInputStream(new StringInputStream("MyStream"), 8));
s3Client.putObject(request, RequestBody.empty());Version 2.x has similarly expanded the type conversions available when downloading data from Amazon.
S3Client s3Client = S3Client.create();
GetObjectRequest request = GetObjectRequest.builder()
.bucket("my-bucket")
.key("my-key")
.build();
s3Client.getObject(request, ResponseTransformer.toOutputStream(new ByteArrayOutputStream()));
s3Client.getObject(request, ResponseTransformer.toFile(Paths.get("MyFile.out")));
InputStream stream = s3Client.getObject(request, ResponseTransformer.toInputStream());
byte[] bytes = s3Client.getObject(request, ResponseTransformer.toBytes()).asByteArray();
String string = s3Client.getObject(request, ResponseTransformer.toBytes()).asUtf8String();
ByteBuffer byteBuffer = s3Client.getObject(request, ResponseTransformer.toBytes()).asByteBuffer();Amazon S3 client
To provide SDK support for the many services that AWS owns, the AWS SDKs make extensive use of code generation. In 1.11.x, all service clients are generated, except for the S3 client. This frequently results in a disconnect between how non-Java AWS SDKs and IAM policies refer to an S3 operation (e.g., DeleteBucketReplicationConfiguration), and how the Java AWS SDK would refer to that same operation (DeleteBucketReplication). This made creating IAM policies and switching to other SDKs difficult, because the equivalent string was not always well documented.
In 2.x, S3 is generated like every other service, ensuring the operation names, inputs, and outputs will always match those of other SDKs and IAM.
What’s missing in the AWS SDK for Java 2.x
As mentioned, not every feature in 1.11.x has made it to 2.x yet. We recommend using 2.x if you would benefit from the new features and improvements, and using 1.11.x if you would benefit from the features not yet in 2.x. Both versions of the SDK can be used at the same time, allowing you to pick and choose which parts you want to use from each SDK version.
The following 1.11.x features are not in 2.x yet:
- High-level libraries
- Waiters
- SDK metric publishing
- Progress listeners
Further reading
Get started with our AWS SDK for Java 2.x Developer Guide.
See what it takes to migrate your existing 1.11.x application with our 1.11.x to 2.x Migration Guide.
Get an exact list of what’s changed by using the 1.11.x to 2.x Changelog.
Contact us
Check out the 2.x source in GitHub.
Come join the AWS SDK for Java 2.x community chat on Gitter.
Articulate your feature requests or upvote existing ones on the GitHub Issues page.