Shippable Case Study

Shippable is a platform providing hosted continuous integration, testing, and deployment from repositories such as GitHub and Bitbucket. The company helps developers and operations teams deliver software from application creation all the way to the end user with full automation and complete control over the process. Shippable’s platform helps eliminate obstacles and challenges from a repeatable Continuous Integration/Continuous Delivery & workflow, so developers can focus on delivering high quality code.

The Shippable platform consists of two parts:

• Continuous Integration (CI), enabling developers to build and test their repositories for every code commit or pull request and get instant feedback.
• Continuous Delivery (CD) pipelines, automating the flow of an application from source control to production. Using these deployment pipelines, developers can easily deploy containerized applications to container services such as Amazon EC2 Container Service (Amazon ECS).
• Shippable integrates with popular tools and services, such as build/test tools, source control providers, Docker registries, cloud providers, and container services.
  

• The company built its CICD platform with a microservices architecture using Docker containers on Amazon EC2, which they managed using a custom-made cluster management tool.
• The initial solution used a hardcoded service discovery solution, making it hard to scale and manage the configuration.
• The engineering team spent 60-80 percent of its time maintaining the infrastructure and orchestrating containers, taking time away from implementing and delivering features for the platform itself.
• Managing the monitoring infrastructure with an extensive set of log aggregators and Elasticsearch was another time consuming task.
• Shippable evaluated other open-source cluster management solutions such as Apache Mesos and Kubernetes, but found that these solutions would still require a significant amount of engineering time to operate and scale the cluster.
  

• Started using Amazon ECS, a scalable, container management service that provides cluster management and container orchestration.
• Each microservice is defined as an ECS service with an Elastic Load Balancing load balancer; the Amazon ECS service scheduler manages multiple copies of each microservice across the ECS cluster.
• Amazon ECS and Amazon CloudWatch handle all of the infrastructure logging, with minimal effort from the engineering team.
• Shippable Docker images are stored on Amazon EC2 Container Registry (Amazon ECR), a fully-managed Docker container registry that makes it easy to store, manage, and deploy Docker container images.
  

• Amazon ECS eliminates the need to install software and operate servers; this reduces overall management needs for running containers at scale and helps speed time to market.
• The engineering team now spends only 20 percent of its time maintaining the infrastructure, and has much more time to focus on implementing new features.
• Using Amazon ECS enables Shippable to deliver features to its customers faster than before, increasing the number of deployments from once a week to multiple times a day.
• With AWS Identity Access Management (IAM) policies for Amazon ECR, the company is able to add a layer of security that ensures only approved individuals can push and pull production images in the production cluster.