Secure data acquisition and control of DERs and grid devices using ASE/Kalkitech Data Hub on AWS

Industry Context

In the utility industry, there is a wide range of devices and supervisory control and data acquisition (SCADA) systems that operate in an on-premises environment. With the growth of renewables and distributed energy resources (DERs), electric utilities need to change their energy planning models and operations to maintain reliability and stability of the bulk power system. This requires complex modeling using data from DERs, SCADA systems, weather services, and advanced metering infrastructures (AMIs) to run power flow and other computations in near real time to provide situational awareness of the state of the grid. Processing data from multiple sources in near real time requires high-speed computations and scalable compute resources that use the latest CPUs/GPUs along with other technologies, such as parallel clusters, data lakes, and the latest machine learning models, which are all offered by Amazon Web Services (AWS).

So, how do you get your SCADA, AMI, and DER data to AWS to hydrate a data lake or stream the data to applications in near real time? How do you get a distributed energy resources management system (DERMS) software to control batteries that are behind the meter or at a utility’s substation? How do you verify that the communication between your software and the devices is secure? Most devices used by the electric utility industry communicate using industry-specific protocols, which means you would have to write code to read data from the devices, make sure secure channels are used, convert and harmonize the data to a standardized format, and then write code using AWS APIs for Amazon Simple Storage Service (Amazon S3)—an object storage service offering industry-leading scalability, data availability, security, and performance—and Amazon Kinesis—a fully managed service that cost-effectively processes and analyzes streaming data at any scale—to hydrate data lakes and/or add to a data stream. This process can be a lot of work for electric utilities that are not software builders and even for independent software vendors (ISVs) who want to focus on the business functionality of their products rather than the underlying plumbing.

To resolve this issue, ASE/KalkitechKalkitech, an AWS Partner, designed and built the Kalki.io Data Hub as a cloud-based service that can securely communicate with almost any device or SCADA system across multiple protocols and can seamlessly pass the data to AWS services.

Solution Architecture

To address industry needs, ASE/Kalkitech’s solution meets the following criteria:

1. Supports protocols used by the SCADA or operational technology (OT) system and other devices, because these protocols can vary by market, maker, and model
2. Facilitates seamless availability, even across geographies, with automatic failover and recovery
3. Offers end-to-end encryption of data in transit and at rest
4. Facilitates effortless integration with AWS services, helping customers to focus on data consumption rather than on concerns about data acquisition and storage

Kalki.io Data Hub is capable of communicating with devices using industry protocols, and ASE/Kalkitech took the next step of integrating Data Hub with AWS services. With this integration, customers can decide if a certain data tag will go to Amazon S3 to hydrate a data lake or if it will stream to Amazon Kinesis by simple user interfaces.

Figure 1 illustrates how Kalkitech architected security and high availability of the Kalki.io Data Hub. High availability is established by deploying Data Hub across two different Availability Zones (AZs) behind an AWS Network Load Balancer (NLB), which facilitates Layer 4 load balancing across servers in the different AZs, long-lasting transmission control protocol (TCP) connections, and static internet protocol (IP) addresses for older devices. ASE/Kalkitech also included the ability to add the Data Hub application to another AWS Region for cross-region failover. AWS AZs offer customers the ability to deploy servers in different data centers within an AWS Region, thus offering high availability for a load-balanced or clustered application. For disaster recovery and even higher levels of availability, customers can deploy their servers across larger geographic distances using different AWS Regions (for example, the East and West Coasts of the United States).

A customer can also choose to use a single region with just two or three servers deployed, thus offering flexibility in deployment options to meet a specific risk tolerance.

Figure 1. Kalki.io Data Hub architecture on AWS

Security for Data Hub does not stop with a highly available architecture. Data Hub communicates with on-premises SCADA systems, OT devices, or DER sites, and it is important that this communication is fast, secure, and reliable. On AWS, these requirements can be met using AWS Global Accelerator, a networking service that helps you improve the availability, performance, and security of your public applications. AWS Global Accelerator provides two global static public IP addresses that act as a fixed entry point to an application’s endpoints, such as NLBs. With this approach, any customer using the Kalki.io Data Hub can establish high levels of availability across multiple geographic regions. This service can also help achieve high-speed data transfer from on-premises SCADA systems to the Data Hub in the cloud and can facilitate rapid failover from one AWS Region to another.

Alternatively, a customer can use AWS Direct Connect, the shortest path to AWS resources. While in transit, network traffic remains on the AWS global network and never touches the public internet. This reduces the chance of hitting bottlenecks or unexpected increases in latency.

As an added layer of security, ASE/Kalkitech offers the option of using Kalki.io Network Proxy, an on-premises security application that uses x.509 certificates and mutual TLS to establish a secure tunnel between a utility’s data center or DER sites and the Kalki.io Data Hub in the cloud, thus offering complete encryption of data in transit.

On the AWS side, inbound traffic is received by an AWS NLB that is configured to receive data on specified ports with a security group that further applies both port and source IP address restrictions. The servers themselves sit in a private subnet; they are not accessible from the internet and are not even directly accessible for interactive access from the customer’s on-premises data center. Data is further protected by the use of virtual private cloud (VPC) endpoints to communicate with AWS services such as Amazon S3 and Amazon Kinesis. VPC endpoints help Kalki.io Data Hub to privately connect to Amazon S3 and Amazon Kinesis without the need for public IP addresses, thus keeping the data traffic on the AWS network.

For maintenance of the servers a customer’s engineer can access the servers using AWS Session Manager or EC2 Instance Connect Endpoint (EIC Endpoint). EIC Endpoint eliminates the cost and operational overhead of maintaining bastions/jump boxes. EIC Endpoint combines AWS Identity and Access Management (IAM) based access controls to restrict access to trusted principals, with network based controls such as Security Group rules, and provides an audit of all connections via AWS CloudTrail, helping customers improve their security posture.

In addition, customers using Kalki.io Data Hub on AWS can use various AWS services for management and governance of their environment and services for security, identity, and compliance. For example, customers should use AWS CloudTrail to log all actions in their accounts, and AWS Key Management Service (AWS KMS)—a service that lets you create, manage, and control cryptographic keys across your applications and AWS services—can encrypt virtually all data on Data Hub, Amazon Kinesis streams, and Amazon S3 buckets.

Conclusion

Kalki.io Data Hub is a highly available, cloud-based application designed to make it easier for utilities and ISVs to build systems that require near-real-time SCADA data or data from grid devices.

Kalki.io Data Hub on AWS meets your needs for the highly available and secure protocol conversion and data communication required for building applications and solutions such as VPP/DERMS; data lakes; advanced analytics; simulations; fault location, isolation, and service restoration (FLISR); and near-real-time situational awareness, to name a few.

If you have ideas or feedback, please reach out to us or leave a comment. If you are a utility, a system integrator, or an ISV looking to learn more about how you can transform and build the future of your energy business, visit AWS for Energy.