Introducing AWS Panorama – Improve your operations with computer vision at the edge

Yesterday at AWS re:Invent 2020, we announced AWS Panorama, a new machine learning (ML) Appliance and SDK, which allows organizations to bring computer vision (CV) to their on-premises cameras to make automated predictions with high accuracy and low latency. In this post, you learn how customers across a range of industries are using AWS Panorama to improve their operations by automating monitoring and visual inspection tasks.

For many organizations, deriving actionable insights from onsite camera video feeds to improve operations remains a challenge, whether it be increasing manufacturing quality, ensuring safety or operating compliance of their facilities, or analyzing customer traffic in retail locations. To derive these insights, customers must monitor live video of facilities or equipment, or review recorded footage after an incident has occurred, which is manual, error-prone, and difficult to scale.

Customers have begun to take advantage of CV models running in the cloud to automate these visual inspection tasks, but there are circumstances when relying exclusively on the cloud isn’t optimal due to latency requirements or intermittent connectivity. For these reasons, CV processed locally at the edge is needed, which makes the data immediately actionable. Some customers have begun exploring existing capabilities for CV at the edge with enterprise cameras, but find that many cameras lack the capability to perform on-device CV, or offer only simple, hard-coded ML models that can’t be customized or improved over time.

AWS Panorama

AWS Panorama, is an ML Appliance and SDK (software development kit) that enables you to add CV to your existing on-premises cameras or use new AWS Panorama-enabled cameras for edge CV, coming soon from partners like Axis Communications and Basler AG.

With AWS Panorama, you can use CV to help automate costly visual inspection tasks, with the flexibility to bring your own CV models, such as those built with Amazon SageMaker, or use pre-built models from AWS or third parties. AWS Panorama removes the heavy lifting from each step of the CV process by making it easier to use live video feeds to enhance tasks that traditionally required visual inspection and monitoring, like evaluating manufacturing quality, finding bottlenecks in industrial processes, assessing worker safety within your facilities, and analyzing customer traffic in retail stores.

The AWS Panorama Appliance

The AWS Panorama Appliance analyzes video feeds from onsite cameras, acting locally on your data in locations where network connectivity is intermittent, and generates highly accurate ML predictions within milliseconds to improve operations.

The AWS Panorama Appliance, when connected to a network, can discover and connect to existing IP cameras that support the ONVIF standard, and run multiple CV models per stream. Your cameras don’t need any built-in ML or smart capabilities, because the AWS Panorama Appliance provides the ability to add CV to your existing IP cameras.

With an IP62 rating, the AWS Panorama Appliance is dust proof and water resistant, making it appropriate for use in harsh environmental conditions, enabling you to bring CV to where it’s needed in industrial locations.

The AWS Panorama Device SDK

The AWS Panorama Device SDK is a device software stack for CV, sample code, APIs, and tools that will support the NVIDIA Jetson product family and Ambarella CV 2x product line. With the AWS Panorama Device SDK, device manufacturers can build new AWS Panorama-enabled edge devices and smart cameras that run more meaningful CV models at the edge, and offer you a selection of edge devices to satisfy your use cases. For more information, refer to the AWS Panorama SDK page.

Customer stories

In this section, we share the stories of customers who are developing with AWS Panorama to improve manufacturing quality control, retail insights, workplace safety, supply chain efficiency, and transportation and logistics, and are innovating faster with CV at the edge.

Manufacturing and industrial

AWS Panorama can help improve product quality and decrease costs that arise from common manufacturing defects by enabling you to take quick corrective action. With the AWS Panorama Appliance, you can run CV applications at the edge to detect manufacturing anomalies using videos from existing IP camera streams that monitor your manufacturing lines. You can integrate the real-time results with your on-premises systems, facilitate automation, and immediately improve manufacturing processes on factory floors or production lines.

“For packages at Amazon Fulfillment Centers to be successfully packed in a timely manner, the items must first be inbounded into our structured robotic field via an efficient stow process. Items are stowed individually into different bins within each pod carried by our robotic drive units. Today, we use ML computer vision action detection models deployed on SageMaker (in the cloud) to accurately predict the bin in which each item was placed. AWS Panorama gives us the flexibility to run these same models in real time on edge devices, which opens the door to further optimize the stowing process.”

– Joseph Quinlivan, Tech VP, Robotics & Fulfillment, Amazon

“Many unique components go into each guitar and each instrument is carefully crafted. With AWS Panorama and help from the Amazon Machine Learning Solutions Lab, we can have real-time visibility into metrics for the first time such as how long it takes to manufacture different types of guitars, and how much time each process requires. We can use these metrics to spot bottlenecks, optimize efficiency, and improve overall operations.”

– Michael Spandau, SVP Global IT at Fender.

Reimagined retail insights

In retail environments, the AWS Panorama Appliance enables you to run multiple, simultaneous CV models on the video feeds from your existing onsite cameras. Applications for retail analytics, such as for people counting, heat mapping, and queue management, can help you get started quickly. With the streamlined management capabilities that AWS Panorama offers, you can easily scale your CV applications to include multiple process locations or stores. This means you can access insights faster and with more accuracy, allowing you to make real-time decisions that create better experiences for your customers.

“We want to use computer vision to better understand consumer needs in our stores, optimize operations, and increase the convenience for our visitors. We plan to use AWS Panorama to deploy different computer vision applications at our stores and experiment over time to strengthen our customer experience and value proposition.”

– Ian White, Senior Vice President, Strategic Marketing and Innovation, Parkland

“TensorIoT was founded on the instinct that the majority of the ‘compute’ is moving to the edge and all ‘things’ are becoming smarter. AWS Panorama has made moving computer vision to the edge much easier, and we’ve engaged with Parkland Fuel to use AWS Panorama to gather important retail analytics that will help their business thrive.”

– Ravikumar Raghunathan, CEO, TensorIoT

“Pilot.AI solves high-impact problems by developing computationally efficient algorithms to enable pervasive artificial intelligence running at the edge. With AWS Panorama, customers can rapidly add intelligence to their existing IP cameras and begin generating real-time insights on their retail operations using Pilot.AI’s high-performance computer vision models.”

– Jon Su, CEO, Pilot AI

Workplace safety

AWS Panorama allows you to monitor workplace safety, get notified immediately about any potential issues or unsafe situations, and take corrective action. AWS Panorama allows you to easily route real-time CV application results to AWS services such as Amazon Simple Storage Service (Amazon S3), Amazon Kinesis Video Streams, or Amazon CloudWatch and gather analytics. This means you can make improved data-based decisions to enhance workplace safety and security for your employees.

“Bigmate is focused on critical risk management solutions that leverage computer vision to help organizations improve workplace health and safety. Whether it’s keeping your people out of the way of hazardous equipment or ensuring they have the proper Personal Protective Equipment (PPE), with AWS Panorama we can rapidly deploy a suite of apps using your existing CCTV cameras that provide real-time notifications to avoid critical events while providing you the data you need to drive a safety-first culture.”

– Brett Orr, General Manager Chairman, Bigmate

“Organizations are facing unprecedented demand to transform and secure their physical spaces. With Accenture’s Rhythm.IO, we’re focused on helping customers create maximal situational awareness and safer environments, whether for shopping, travel, or public safety, by fusing together operational data and multi-sensor inputs with computer vision insights from AWS Panorama.”

– Matthew Lancaster, Managing Director, Accenture

“Construction zones are dynamic environments. At any given time, you’ve got hundreds of deliveries and subcontractors sharing the site with heavy equipment, and it’s changing every day. INDUS.AI is focused on delivering construction intelligence for general contractors. Computer vision is an especially valuable tool for this because of its ability to handle multiple tasks at once. We are looking forward to delivering real-time insights on jobsite management and safety in a SaaS-like experience for AWS Panorama customers.”

– Matt Man, CEO, INDUS.AI

Supply chain efficiency

In manufacturing and assembly environments, AWS Panorama can help provide critical input to supply chain operations by tracking throughput and recognizing bar codes, labels of parts, or completed products. Customers in an assembly plant, for example, might want to use AWS Panorama to automatically identify labels and bar codes of goods received at certain identification points, for automatic booking of goods into a warehouse management system.

“Computer vision helps us innovate and optimize several processes, and the applications are endless. We want to use computer vision to assess the size of trucks coming to our granaries in order to determine the optimal loading dock for each truck. We also want to use computer vision to understand the movement of assets in our plants to remove bottlenecks. AWS Panorama enables all of these solutions with a managed service and edge appliance for deploying and managing a variety of local computer vision applications.”

– Victor Caldas, Computer Vision Capability Lead, Cargill

“Every month, millions of trucks enter Amazon facilities, so creating technology that automates trailer loading, unloading, and parking is incredibly important. Amazon’s Middle Mile Product and Technology (MMPT) has begun using AWS Panorama to recognize license plates on these vehicles and automatically expedite entry and exit for drivers. This enables a safe and fast visit to Amazon sites, ensuring faster package delivery for our customers.”

– Steve Armato, VP Middle Mile Product and Technology, Amazon

Transportation and logistics

AWS Panorama allows you to process data to improve infrastructure, logistics, and transportation; get notified immediately about any potential issues or unsafe situations; and implement appropriate solutions. AWS Panorama Appliance allows you to easily connect to existing network cameras, process videos right at the edge, and collect metrics for real-time intelligence, while complying with regulatory requirements on data privacy such as processing data locally without storing the videos locally or transmitting videos to the cloud. This means you can get the information needed to provide improved services to your personnel.

“Siemens Mobility has been a leader for seamless, sustainable, and secure transport solutions for more than 160 years. The Siemens ITS Digital Lab is the innovation team in charge of bringing the latest digital advances to the traffic industry, and is uniquely positioned to provide data analytics and AI solutions to public agencies. As cities face new challenges, municipalities have turned to us to innovate on their behalf. Cities would like to understand how to effectively manage their assets and improve congestion and direct traffic. We want to use AWS Panorama to bring computer vision to existing security cameras to monitor traffic and intelligently allocate curbside space, help cities optimize parking and traffic, and improve quality of life for their constituents.”

– Laura Sanchez, Innovation Manager, Siemens Mobility ITS Digital Lab

How to get started

You can improve your business operations with AWS Panorama in three steps:

1. Identify the process you want to improve with computer vision.
2. Develop CV models with SageMaker or use pre-built models from AWS or third parties. If you need CV expertise, take advantage of the wealth of experience that the AWS Panorama partners offer.
3. Get started now with the preview and evaluate, develop, and test your CV applications with the AWS Panorama Appliance Developer Kit.

About the Authors

Banu Nagasundaram is a Senior Product Manager – Technical for AWS Panorama. She helps enterprise customers to be successful using AWS AI/ML services and solves real world business problems. Banu has over 11 years of semiconductor technology experience prior to AWS, working on AI and HPC compute design for datacenter customers. In her spare time, she enjoys hiking and painting.

Jason Copeland is a veteran product leader at AWS with deep experience in machine learning and computer vision at companies including Apple, Deep Vision, and RingCentral. He holds an MBA from Harvard Business School.