Contribution: Introduction of efforts aimed at cloud adoption of smart meter systems by Kansai Transmission and Distribution, Inc. (Part 2) — Second half

This article is the second half of efforts aimed at cloud adoption of smart meter systems by Kansai Transmission and Distribution, Inc. We received a contribution from Mr. Yasuo Matsuura, an executive officer. The introduction will be divided into 2 parts: the first half and the second half. This article will be in the latter half. Check out this link for the first half.

Also, for the first contribution, please refer to “Contribution: Introduction of efforts aimed at cloud adoption of smart meter systems by Kansai Transmission and Distribution, Inc. (First half).”

4. System Architecture

Next, we’ll give you an overview of the next generation smart meter system architecture we’re considering.

When implementing a loosely coupled architecture in line with the development policy described above, cooperation between subsystems is based on loose coupling. Specifically, in collaboration between subsystems from HES to MDMS and from MDMS to a meter data utilization platform that centrally manages data, it was designed so that data transfer with high elasticity and high real-time can be realized simultaneously while asynchronously linking between systems using Amazon SQS. By constructing a mechanism based on SQS, it is possible to receive meter reading values that occur in large quantities on a regular basis and event information that is transmitted in large quantities during power outages or restoration while buffering, can be linked to subsequent processing at high speed, and has an architecture that enables near real-time processing that maintains elasticity against large amounts of burst traffic.

Also, within subsystems based on loosely coupled architectures, we will promote the use of microservices. Business logic is reviewed, functions are broken down, etc., using AWS Lambda etc. for usage calculation and instrument event processing, etc., a package-like application execution environment is realized in a container environment based on Amazon ECS and AWS Fargate, and then a loosely coupled architecture is built by linking APIs with Amazon S3, Amazon DynamoDB, etc. For future implementation of new functions for which the current system has not been decided, we will achieve flexible and agile development while utilizing serverless and containers with segmented functions.

Keeping in mind improvements in operation and maintainability related to database security and version upgrades, etc., the policy is to set up a system centered on AWS managed services, etc., and along with this, not only S3, but also databases such as Amazon Aurora, Amazon KeySpaces, DynamoDB, and Amazon RDS for PostgreSQL We utilize various managed services, such as Amazon API Gateway for data linking.

Toward future advanced utilization of meter reading value data, equipment information, etc. collected by smart meter systems and its revitalization, we will first centrally manage such information in a data store based on meter data utilization, and then advance develop a data utilization mechanism starting from this while using Lambda, API Gateway, etc. In order to advance data utilization, we plan to create an environment where easy data analysis is possible by linking Amazon QuickSight etc. with this data store, and also utilize AI/ML services from the trial and error stage so that they can be applied to business.
Figure 6 shows an overall overview of the system we are developing.

Figure 6. Architectural Overview of Our Next Generation Smart meter system

In our 15 years of smart meter work, we have faced various challenges. These included, for example, issues that lack flexibility in data utilization, such as overlapping functions between systems and complex collaboration methods etc., issues of economy and future sustainability associated with adopting system vendors’ own OS, commercial databases, and middleware, issues such as system blackboxing and concealment of business processing logic, and further future uncertainty, such as delays in server hardware procurement delivery due to semiconductor exhaustion. We believe that the root cause of these issues is that we were unable to develop under a unified system concept, and this time we are focusing on incorporating our own development concept.

For that, it is necessary to correctly understand the system infrastructure on the AWS cloud, which we will now use as our development platform. In other words, instead of entrusting everything from the application layer to the infrastructure layer supporting it to the system vendor to develop the system, this time, after organizing the cloud infrastructure layer and common functions related to the entire system, such as AWS account management for applications on it, network functions, security measures, etc. as a “common foundation”, then we decide that we will control the whole thing including the development direction of the application layer by managing this common foundation. We believe that by correctly understanding the infrastructure layer and understanding it in depth, we can proceed with the development of applications based on this while firmly cooperating with system vendors. Since system vendors will also be able to focus on application development linked to business, we believe that as a result, it will be possible to secure system quality even more than before, and we will be able to obtain a reliable path for future data utilization.

There are three points of effort for us to correctly understand and manage the cloud infrastructure layer, and to internalize future data utilization.

The first is to split AWS accounts based on AWS’s multi-account strategy. By doing so, we will clarify the scope of responsibility and strengthen security aspects. Specifically, after managing the common infrastructure, AWS accounts are divided for each system vendor, and vendors are required to carry out system construction and maintenance within the account. Furthermore, connection points between systems are minimized and responsibility demarcation points are clarified. Necessary user IDs are also provided from the common infrastructure side. Figure 7 shows how to organize this based on the way of thinking about AWS’s multi-account strategy (*2).

(*2) Decide Your AWS Environment Using Multiple Accounts

In this way, the environments for development, testing and operation are separated and maintained, and the breakdown of responsibilities for each common infrastructure and subsystem within each environment is clearly divided by AWS accounts. As a result, system vendors will able to focus on application system development within the granted account, and they will also be responsible for system operation and maintenance.

Figure 7. Splitting AWS accounts between systems to clarify areas of responsibility and enhance security

The second is to clarify the scope of responsibility between AWS, system vendors, and us, based on the AWS shared responsibility model. Along with this, we manage things that require standardization, such as OS, as a common infrastructure and provide them to vendors. Variations between systems related to OS types and versions can be eliminated, which also leads to a reduction in TCO. The specific image is shown in Figure 8.

Figure 8. Role and responsibility base on AWS shared responsibility model

The third is getting help from AWS Professional Services. In order to achieve full use of the cloud, it is essential to introduce a common infrastructure, and on top of that, I think it is important to cooperate more closely with system vendors. This requires us as stakeholders to correctly understand the system and steadily respond to system expansions and changes associated with environmental changes; in other words, it is essential that we correctly understand AWS, and with the cooperation of AWS Professional Services, we are accelerating studies and acquisition of skills and knowledge. While cooperating with the members of AWS Professional Services from time to time, we are accelerating system development and proceeding with initiatives by receiving comprehensive support, such as PMO support for project promotion and education support for our members, while appropriately understanding a wide range of technical issues and trying to resolve them in a flexible and accurate manner.

6. Conclusion

In this contribution, I introduced the status of efforts for next-generation smart meter systems utilizing AWS, including technical perspectives. Next time, I would like to introduce the cloud shift of the current smart meter system. The 3rd installment has been published as a serialized article below, so please be sure to check it out.

Article #3 “Contribution: Introduction of efforts aimed at cloud adoption of smart meter systems by Kansai Transmission and Distribution Inc. (Part 3) — First half”

Author

This article was translated by AWS Blake Horike, Riho Matsui, and Satoshi Aoyama.