Posted On: Apr 9, 2020
We’re excited to announce Amazon EMR release 6.0.0 with support for new major versions of Hadoop, Hive, HBase, Amazon Linux 2 and support for packaging Spark environment dependencies with Docker.
Spark users can now use Docker images from Docker Hub and Amazon Elastic Container Registry (Amazon ECR) with EMR release 6.0.0 to define environment and library dependencies. Using Docker, you can easily package your Python and R dependencies for individual jobs, avoiding the need to install dependencies on individual cluster hosts. Fore more details on using Docker with EMR 6.0.0, refer to our EMR management guide section on how to Configure Docker and our blog post on how you can simplify your Spark dependency management with Docker.
Hive users can now use Hive Live Long and Process (LLAP) with EMR release 6.0.0, providing a 2x performance speedup over EMR 5.29 with up to 10x improvement on individual Hive TPC-DS queries*. Hive LLAP is a new execution model in Hive that uses persistent daemons with dynamic in-memory caching to speed up query execution. For details on how to enable Hive LLAP, refer to our documentation on Using Hive LLAP and our blog post to see why Apache Hive is 2x faster with Hive LLAP on EMR 6.0.0.
EMR release 6.0.0 provides new major versions of Apache Hadoop 3.2.1, Apache Hive 3.1.2, Apache HBase 2.2.3, Apache Phoenix 5.0.0, and the EMR runtime for Apache Spark 2.4.4 with support for Scala 2.12. EMR release 6.0.0 is built on Amazon Linux 2, and Amazon Corretto JDK 8. Amazon Linux 2 is the latest generation of the Amazon Linux server operating system, providing new system tools like the systemd init system, and the performance tuned Amazon Linux LTS Kernel. Amazon Corretto JDK 8 provides a Java SE certified compatible JDK that includes long-term support, performance enhancements, and security fixes. For more details on all of the EMR applications updated in EMR release 6.0.0, see our release notes.
Amazon EMR release 6.0.0 is now available in all supported regions for Amazon EMR.
You can stay up to date on EMR releases by subscribing to the feed for EMR release notes. Use the icon at the top of the EMR Release Guide to link the feed URL directly to your favorite feed reader.
*Based on 3TB TPC-DS benchmark comparing EMR 5.29.0 with EMR 6.0.0.