Analytics Without Limits: FINRA’s Scalable and Secure Big Data Architecture – Part 2

A guest post by John Brady, CISSP, VP Cyber Security/CISO, Financial Industry Regulatory Authority

The Financial Industry Regulatory Authority (FINRA) oversees more than 3,900 securities firms with approximately 640,000 brokers. Every day, we watch over nearly 6 billion shares traded in U.S. equities markets—using technology powerful enough to help detect fraud, abuse and insider trading. In fact, FINRA processes approximately 6 terabytes of data and 37 billion records on an average day to build a complete, holistic picture of market trading in the U.S. On busy days, the stock markets can generate 75 billion+ records.

Learn how FINRA uses Amazon S3 and herd to create a data lake in the cloud, to separate compute from storage and optimize cost and scalability here.

In order to migrate to the cloud, we worked on overcoming a few initial questions: Is it secure enough? How will security and governance controls be implemented? And can we improve our security posture with cloud infrastructure?

Security and compliance were never afterthoughts. These groups were key stakeholders in the development and migration of our applications. For example, in the design stage, we engaged our internal security, audit and compliance groups and leveraged both the AWS Service Organization Control (SOC) 1 and 2 Reports and the shared responsibility model to help everyone understand and agree on the business proposition and value the cloud brings. By doing this, we were able to demonstrate how the cloud can help reduce risk, remove manual steps, simplify operations, and thus, enhance controls.

In the last four years as we transitioned to the cloud, I have come to realize that as a relatively small organization, we can be far more secure in the cloud and achieve a higher level of assurance at a much lower cost, in terms of effort and dollars invested. We determined that security in AWS is superior to our on-premises data center across several dimensions, including patching, encryption, auditing and logging, entitlements, and compliance.

For example, we have inherent risk mitigations due to AWS’s scale of operations and separation of duties. And, we have access to best-of-breed solutions, like AWS Key Management Service (KMS), that would be difficult, if not impossible, to develop and manage ourselves. Instead of investing millions of dollars to develop it, we can use AWS KMS for $1 per key per month.

Similarly, with Security Groups we can practice micro-segmentation and put each server into a security zone of one. Micro-segmentation dramatically reduces attack surface area and makes it nearly impossible for an attacker to find exploitable weaknesses and expand control within the VPC. New AWS services, such as Amazon EC2 Container Service (ECS) and AWS Lambda, further reduce our security maintenance effort by providing secure environments for our workloads with no effort expended on our part to maintain servers.

Another example is how we have embraced automation. Our app teams are more aware that they need to be secure in the cloud, and we use this awareness to our advantage to automate and become more compliant and secure. Our DevOps staff focuses on the automation and tools to raise the compliance bar and simplify controls. DevOps works with delivery teams to identify and test out new services. We use our development sandbox to try out new services and vet their usability, security, and compliance. Once we understand how we will integrate, we bring in the security teams to vet the service and run assessments. Then automation created by DevOps is used to consistently build fully compliant QC and Prod environments. And AWS APIs have made it possible for us to create our own compliance monitoring tools which continually check for insecure settings in all our cloud resources.

It all comes down to execution to implement in the cloud correctly. If done right, you can have less exposure than in your own data center, and it can amplify what you, as an organization, can achieve. To assist small firms in establishing a cybersecurity program, FINRA has created a Small Firm Cybersecurity Checklist and a Report on Cyber Practices.

Watch the Security Innovation in the Cloud: CIO & CISO Panel session from the AWS Public Sector Summit to see other security and information technology leaders and me discuss the ways we each embrace AWS for security innovation and best practices for cloud security.