How do you maintain a lean and impactful IT operation when both your customer base and your engineering team are growing rapidly? This was the situation facing FinTech FreeAgent in 2018. In the following blog post we’ll explore how FreeAgent remained agile, while scaling their business, by adopting AWS fully-managed services.

The Company

U.K.-based FreeAgent provides award-winning cloud accounting software, specifically designed to meet the needs of small businesses, contractors, freelancers and their accountants. More than 100,000 UK businesses use FreeAgent to manage their business finances. FreeAgent empowers business owners to look after much of the day-to-day bookkeeping admin, which means accountants can spend more time delivering valuable services such as providing tax or business advice.

FreeAgent has won numerous high-profile industry awards, including accolades at the AccountingWEB Software Excellence Awards, British Accountancy Awards, ICB Luca Awards and Computing Technology Product Awards.

In 2018, FreeAgent was acquired by the NatWest Group, where it is run as an operationally independent entity.

The Challenge

In 2018, FreeAgent was growing rapidly, both in terms of number of customers and headcount, and had ambitious plans for the future.

At the time they were operating their own private cloud based on the Joyent Triton platform on SmartOS, running SmartOS zones. This is a technology similar to Docker that allowed them to avoid the overhead of virtualization and achieve bare metal performance, while taking advantage of workload isolation. This enabled them to operate multi-tenant workloads while maintaining security boundaries and preventing noisy neighbor problems.

The infrastructure was deployed to two co-location sites, run on an active-standby architecture, with data constantly replicated to the standby site to provide a disaster recovery solution. This resulted in a complex replication topology consisting of five MySQL database servers per site.

FreeAgent started evaluating the public Cloud as a way of scaling and simplifying their business, and ultimately decided to move their workloads to AWS, as they felt it offered the engineering team the greatest chance of success.

The Ambition

There were multiple factors that motivated FreeAgent’s migration to AWS:

Deliver more impact

Above all, FreeAgent wanted to be as lean as possible. They wanted to ensure that they focused all of their time and energy in delivering impact and spent as little time as possible on toil activities, or re-inventing core platform technologies

Be productive at scale

Ensuring a team of 30 engineers with a few workloads are happy and productive is one thing. Ensuring a team of over 100 engineers, with a growing number of workloads, are productive is an entirely different proposition! FreeAgent wanted to lower the barrier to entry for new teams, to make them autonomous and more involved in operating workloads in production.

Growing customer numbers and features also meant resource and capacity planning was becoming a considerable effort, so moving to a platform with proven scale and the benefits of an on-demand resource model were appealing.

Improve security

Staying secure is a constant challenge; it’s not a project that can be shipped. FreeAgent wanted to continue to improve their security position. They felt that AWS’s shared responsibility model delivered the security they needed, with fewer trade-offs in terms of time and complexity.

Solution Architecture

FreeAgent based their migration around a number of AWS best practices:

Deploying across multiple AWS accounts

FreeAgent opted for a multi-account architecture using AWS Organizations in order to provide strong isolation between workloads. Utilizing multiple accounts limits the blast radius for security incidents, failures and human error.  Each account is owned by a single team and used to run a single workload. This simplifies security, as responsibility for providing access to a particular workload is delegated to the team that owns it. This also enables easier auditing of dependencies and data flow between accounts.

Access to individual AWS accounts is managed via cross account roles that users can only assume after two-factor authentication. Roles are carefully scoped to provide the minimum permissions needed.

FreeAgent also has a few service accounts. For example, the Security account is used to host all of the IAM users. A Shared Services account is used to host shared infrastructure such as the CI/CD pipeline and other services that multiple accounts make use of. The Security Operations account is used to centralize the AWS Security Hub and related services, providing the security team with a single place to view all security findings for all accounts.

In addition to these service accounts, FreeAgent tends to deploy a set of standard accounts for each workload. This consists of staging and production accounts, where staging is an exact mirror of production with, typically, changes to instance sizes and capacity for cost management.

This provides teams a path to production that allows every change to be tested in a safe environment, with minimal risk. This gives a high degree of confidence in production changes.

Use of fully-managed services

FreeAgent has chosen to adopt AWS fully-managed services wherever possible.

Container-based workloads are run using a combination of Amazon Elastic Container Service (ECS) and AWS Fargate. Standardizing on ECS has supported the use of repeatable patterns for logging, monitoring, error handling and secrets management. Additionally, CI/CD is simplified as regardless of the application stack, they only need to build, ship and deploy a container image. FreeAgent appreciates the fully-managed aspect of Fargate (which means that they don’t have any servers to manage).

Amazon EC2 is generally used where ECS does not make sense due to technical constraints. However, the company is increasingly making use of AWS Lambda serverless compute platform.

For example, the main FreeAgent application is written in Ruby on Rails and is deployed to ECS running on AWS Fargate. It runs web front end tasks, and makes use of delayed job for job processing. The application also makes use of scheduled tasks. AWS Lambda functions are used for some specific functions, for example generating thumbnails.

Amazon Aurora provides the database with use of Amazon ElastiCache for in-memory caching. Other services used include: Amazon OpenSearch Service for providing in-application search, AWS Key Management Service (KMS) for key management, Elastic Load Balancing, and Amazon CloudFront as content delivery network (CDN) for delivering static web data.

Operations

Deployment of containers is achieved using a 3^rd-party service: Harness. Monitoring is carried out using a combination of DataDog and Amazon CloudWatch.

Logging is provided by Humio. For logs that hit Amazon Cloudwatch Logs first, AWS Lambda is used to publish to Humio.  AWS FireLens and Amazon Kinesis are used to forward the ECS logs to Humio.

Regardless of application specific architecture, a number of services are deployed across all accounts to manage security and operational requirements. For example: Amazon GuardDuty and AWS Config.

Infrastructure as code

FreeAgent use Terraform to provision all of their infrastructure (using reusable templates).  This allows them to audit and peer review all infrastructure changes, and use a versioned release process.  When a new version of their infrastructure is released, this is deployed to staging and verified before promoting to production.

The change process is managed using Atlantis, which allows them to plan and apply changes to AWS accounts, and to drive the entire process from Github pull requests.

The Outcome

In December 2020, FreeAgent successfully switched their workloads over to AWS and 100k customers were migrated with zero downtime. It’s fair to say that FreeAgent’s ambitions for the migration have been fulfilled.

Making use of AWS fully-managed services has increased the speed with which teams can deliver impact. Being able to forgo developing core platforms allows teams to focus on delivering real business impact much sooner. For example: before the AWS migration, a team had looked at running Kafka on-premises and everything required to operate it at scale (high availability, monitoring, etc). The effort looked daunting. Post-migration, the same team delivered the project using AWS Kinesis in a matter of weeks, with no complicated infrastructure to manage.

The amount of time the teams spend on toil work, such as security patching, has significantly decreased and the burden is shared more widely across the organization.

Application teams now operate autonomously, managing their own infrastructure (both in development and production) on a self-service basis.

The breadth of security features AWS offers has allowed FreeAgent to improve their security posture with minimal effort. Data encryption at rest is often a simple option when provisioning a resource. For example, thorny issues such as key management are now handled by KMS.

FreeAgent is keen to point out that, ultimately, their customers benefit, as FreeAgent can concentrate on building a better product.