What is a Service Mesh?

A service mesh is a dedicated infrastructure layer that manages communications between services in a distributed application. A service mesh intercepts traffic to handle a broad range of functions, including observability, service discovery, logging and tracing, and service retries, without requiring developers to make special changes to containerized microservices.

In modern application architectures, you can build applications as collections of small, independently deployable microservices. Although different teams can build all the services using different languages and tools, the individual services must communicate reliably and securely for the application to function properly.

Application stability depends on the speed, resiliency, and security of the communication network between these services. Without a service mesh layer, developers must use multiple tools and services for encryption and monitoring, with integration management across the stack. Instead, a service mesh provides a unified layer for these types of operations.

Here are two primary drivers of service mesh adoption.

Read about microservices »

Service-level observability

As organizations deploy more workloads and services for distributed applications, it becomes increasingly challenging to understand how everything works together. A service mesh solves this by generating metrics, logs, and traces for all traffic. This traffic observability gives service teams better visibility into upstream and downstream dependencies and communication performance at the application layer.

Service-level control

Administrators should have fine-grained control over how traffic flows between services. A service mesh allows you to configure advanced traffic behavior independent of application logic. This means that a failure in one service won't cascade and down the entire application.

A service mesh provides a centralized, dedicated infrastructure layer that handles the intricacies of service-to-service communication within a distributed application.

Here are some specific service mesh benefits.

Service discovery

Service meshes can automate service discovery, reducing the operational burden of managing a large number of service endpoints. There are several ways to accomplish this. For example, Amazon ECS Service Connect abstracts its services registry and allows you to assign hard-coded, logical names to every service (e.g., db-service). That means developers can use the service name in new code without worrying whether the underlying IP address changes.

Load balancing

Service meshes use various algorithms to intelligently distribute requests across multiple service instances, such as round-robin, least-connections, or weighted load balancing. In contrast to traditional software architecture, where central load balancers are most common, a service mesh performs load balancing client-side using its proxies. This can eliminate centralized load balancing bottlenecks and allow for advanced routing.

Traffic management

Service meshes provide advanced traffic management capabilities, which provide fine-grained control over request routing and traffic behavior. Here are a few examples.

Traffic splitting

You can divide incoming traffic between different service versions or configurations. The service mesh directs some traffic to the updated version, which allows for a controlled and gradual rollout of changes. This provides a smooth transition and minimizes the impact of changes.

Request mirroring

Also called "shadowing." You can duplicate traffic to a test or monitoring service for analysis without impacting the primary request flow. When you mirror requests, you gain insights into how the service handles particular requests without affecting the production traffic. In Kubernetes on AWS, this is available through Istio.

Canary deployments

You can direct a small subset of users or traffic to a new service version, while most users continue to use the existing stable version. By limiting exposure to new production code, you can test the new version's behavior and performance in a real-world environment.

Security features

Service meshes provide secure communication features such as mutual transport layer security (mTLS) encryption, which allows for identity verification in service-to-service communication. It helps improve automation and enforcement in your data confidentiality and integrity by encrypting traffic, and also supports a Zero Trust Security model. You can also enforce authorization policies to control which services access specific endpoints or perform specific actions.

Observability

Service meshes observability tools provide deep insights into your services' health, performance, and behavior. They can also support troubleshooting efforts and performance optimization. Here are examples of observability features within AWS service mesh services:

• Collect metrics such as latency, error rates, and resource utilization
• Perform distributed tracing to observe the requests' complete path and timing across multiple services
• Capture service events in logs for auditing, debugging, and compliance purposes

A service mesh removes the logic governing service-to-service communication from your application code and abstracts it into its own infrastructure layer. It typically uses a network proxy system to intercept, route, and secure traffic. These proxies are sometimes referred to as "sidecars" because they run separately but logically next to each service container, intercepting all traffic moving to and from them.

There are two main components of service mesh architecture: the control plane and the data plane.

Data plane

A service mesh features a data plane for handling data. If your service mesh uses sidecar proxies, they exist here. Service mesh functionality, such as load balancing, service discovery, and traffic routing, is also implemented in the data plane. When a service wants to communicate with another service, the sidecar proxy:

1. Intercepts the request
2. It encapsulates the request and forwards it with a separate network connection
3. It establishes a secure and encrypted channel between the source and destination proxies

The sidecar proxies handle low-level messaging between services. They also implement features such as circuit breaking and request retries to enhance resiliency.

Control plane

The control plane acts as the central management and configuration layer of the service mesh.

Administrators define and configure their services in the control plane. For example, they can specify service endpoints, routing rules, load-balancing policies, and their security settings. After they define this configuration, the control plane distributes the necessary information to the service mesh's data plane.

The proxies use the configuration information to decide how to handle incoming requests. They can also receive configuration changes and adapt their behavior dynamically. You can make real-time changes to the service mesh configuration without service restarts or disruptions.

Service mesh technology typically includes the following capabilities in the control plane:

• Service registry that keeps track of all services within the service mesh
• Automatic discovery of new services and removal of inactive services
• Telemetry configuration for metrics, logs, and distributed tracing

Istio is an open-source service mesh project designed to work primarily with Kubernetes. Kubernetes is an open-source container orchestration platform for deploying and managing containerized applications at scale.

Istio's service mesh control plane components run as Kubernetes workloads. It uses a Kubernetes Pod as the basis for its sidecar proxy design. Each Pod is a tightly coupled set of containers that share one IP address.

Istio's layer 7 proxy runs as another container in the same network context as the main service. From there, it can perform traffic control by intercepting, inspecting, and manipulating all traffic heading through the Pod before it becomes outbound network traffic. The primary container does not need any alteration or even knowledge that this is happening.

Read about Amazon Elastic Kubernetes Service (Amazon EKS) »

Here are some common service mesh challenges with self-managed open-source platforms such as Istio, Linkerd, and Consul.

Complexity

Service meshes provide capabilities that can help coordinate and monitor a microservices architecture. However, they also introduce additional infrastructure components, configuration requirements, and deployment considerations. Using them can be a steep learning curve, requiring developers and operators to gain expertise with the specific service mesh implementation. It takes time and resources to train teams. An organization must make sure operations teams have the necessary knowledge to understand the intricacies of service mesh architecture and configure it effectively.

Operational overheads

Service meshes can require more overhead to deploy, manage, and monitor the data plane proxies and control plane components. For instance:

• Ensure the availability and scalability of the service mesh infrastructure
• Monitor the health and performance of the proxies
• Handle upgrades and compatibility issues

It's essential to carefully design and configure the service mesh to minimize any performance impact on the overall system.

Integration challenges

A service mesh must integrate seamlessly with existing infrastructure to function effectively. This includes container orchestration platforms, networking solutions, and other tools in your technology stack.

It can be challenging to maintain compatibility with other components in complex and diverse environments. Ongoing planning and testing are required to change your configuration formats and dependencies. The same is true if you need to upgrade to new versions anywhere in the stack.

AWS offers a range of services that can help with creating the right service mesh solution for your requirements:

• Amazon Elastic Container Service (ECS) Service Connect provides service mesh solution capabilities integrated directly into Amazon ECS. Unlike a standalone service mesh, Service Connect is a feature of ECS, meaning you pay for the resources your tasks use, but not for a separate mesh control plane. It automatically deploys and manages Envoy proxies as sidecars.
• Amazon VPC Lattice provides service-to-service connectivity, security, and monitoring. It is a service networking layer designed specifically for resources within your virtual private cloud. It is well-suited for managing service-to-service connectivity if your architecture spans multiple compute types or requires complex cross-account permissions.
• AWS supports Istio deployments on Amazon Elastic Kubernetes Service (Amazon EKS). Within EKS, the EKS Pod Identity agent and standard EKS add-ons allow you to run a standard open-source service mesh with AWS-native security integration.

Get started with service meshes on AWS by creating a free account today.