Business Productivity

Automate moderation and sentiment analysis with Amazon Chime SDK messaging

Live event organizers need to communicate in large streaming events, live streamed product demonstrations, and keynote speeches at a conferences over multiple modalities including video and chat, but existing chat solutions often do not scale to support tens of thousands of participants. Amazon Chime SDK messaging is secure, persistent, and can scale up to 100k members in a single chat channel. In this post, you will learn how to use the Amazon Chime SDK to create a chat application that can be used alongside a live-stream broadcast. This application includes commonly requested features for this use case, including automated moderation, message effects, and sentiment analysis.

For this walk-through, you should have the following pre-requisites:

In this post, I will provide a brief overview of the components of the moderated chat demo app, then provide steps to deploy the demo, followed by a description of each of the app features.

Note: Deploying and using the demo created in this post will incur charges. If you wish to discontinue use and stop incurring charges, there are steps to delete the demo and all resources at the end of this post.

Application Architecture

This application includes a basic chat interface for a single channel with unauthenticated users. If you want to apply the backend integrations shown here with a feature-rich chat front-end, you can take the same backend functionality and apply it to the chat demo application from the messaging launch blog.

There are six key components of this architecture:

  1. Amazon Chime App Instance – A container for Amazon Chime SDK messaging. Settings such as retention policies and streaming export of messaging data through Amazon Kinesis are set at the app instance level. Within the App Instance, you have:
    1. App Instance Admin – This is the user used in the AWS Lambda functions to perform moderation and administration actions such as sending messages warning about use of profanity, redacting sensitive content, and sending control messages with sentiment.
    2. A single channel – With this simplified UI, a single channel is created, and only that channel is displayed in the interface.
    3. Users – Users are created by the user creation Lambda after they enter their name
  2. Amazon API Gateway – Connects the client to the user creation Lambda
  3. Create Chat User Lambda – Creates a user and adds them to the chat channel
  4. Amazon Kinesis Data Stream – Exports chat data into the Lambda function for processing.
  5. Message Processing Lambda – Processes messages, scanning for profanity, and integrating with Amazon Comprehend
  6. Amazon Comprehend – Amazon Comprehend is used for sentiment analysis and detection of personally identifiable information (PII)

Deploying the Moderated Chat Demo App

Step 1. Create AWS resources

  1. Sign in to the AWS Management Console with your primary account. Switch to the us-east-1 (N. Virginia) Region. Note: The AWS CloudFormation template in this section needs to be launched in US east (N. Virginia) Region.
  2. Click on the Launch Stack image below this sentence to launch an AWS CloudFormation template that will setup the required infrastructure for the Amazon Chime SDK chat demo app in your AWS account.
    Launch stack
  3. On the Create Stack page, click Next.
  4. On the Specify Details page, enter the stack name:
    DemoName-ChimeModeratedChatDemo
  5. Choose Next, and then Next on the Configure stack options page.
  6. On the Review page, check the I acknowledge that AWS CloudFormation might create IAM resources check box. Then click Create.
  7. Creating the stack generates 4 outputs- adminUserArn, appInstanceArn, channelArn, createUserApiGatewayURL. Note these values for the outputs to use to configure the app in the next step.
    amazon chime moderated chat demo app CF outputs

Step 2. Deploy the moderated chat demo app locally

In this section, you deploy a simple web app written in the React framework in your local environment. This app demonstrates the Amazon Chime SDK messaging features. Once you set up the app locally, you are able to get started.

  1. Clone the project from GitHub.
    git clone https://github.com/aws-samples/amazon-chime-sdk.git
  2. Run the following commands to navigate to the root folder of the moderated chat demo app:
    cd ./amazon-chime-sdk/apps/moderated-chat-and-sentiment-analysis
  3. Run the following commands to navigate to install dependencies from the root folder:
    npm install
  4. Open src/Config.js with the editor of your choice. Add the following configuration to it:
    const appConfig = {
    createUserApiGatewayURL: “<API Gateway URL from CloudFormation Outputs>“,
    appInstanceArn: '<appInstanceArn from CloudFormation Outputs>',
    adminUserArn: '<adminUserArn from CloudFormation Outputs>',
    channelArn: '<channelArn from CloudFormation Outputs>'
    };"
    export default appConfig;
  5. Once the configuration for the application is entered, run the following commands in the root folder
    npm run start
    Note: If you get an error compiling, try running npm install react-is
  6. Open your browser and navigate to https://localhost:9000 to start testing.
    1. Enter your name to join the channel
    2. Send a message with the words ‘profanity’, ‘swear’, ‘obscenity’, or ‘heck’ to test profanity filtering. This list is found in the filteredWords variable in the index.js file for the AWSSDKChimeModeratedChatDemo-MessageModerator AWS Lambda function.
  7. Enter PII, like your phone number, in a message and it should be redacted.
  8. Enter a positive word like ‘awesome’ in a message and you should see the positive sentiment increase and emojis fly over the screen.

In the next few sections, I will go into a bit more detail about how each of these features work.

Basic Authentication

The moderated chat demo app introduces a simple process for granting users access to a channel. This is great for use cases where you want users to be able to access a channel quickly, even when they don’t have a user account.

join screen

While this example is simple, it begins to show the flexibility of authentication in Amazon Chime SDK messaging. In this flow, you as a developer control exactly how users access a channel and can scope down access as you see fit by simply editing the IAM policy for the user role.

For example, in this flow you may want to tag this user as a guest to limit the user’s access to a single channel with a condition in the IAM policy, rather than any channel in the app instance. You can take this even further and limit what actions the user can take in the channel such as preventing them from listing the members of the channel or preventing them from sending messages making the channel read only.

Because Amazon Chime SDK messaging works with almost any identity provider, you can replace the AWS Lambda function shown in the diagram with your own IdP and eliminate guest access altogether, requiring all users to sign in.

Processing messages to detect PII or profanity

Moderation of user-generated content is a critical need for many chat use cases. Corporations need to prevent the accidental sharing of sensitive information, entertainment and social applications need to ensure community guidelines are met, and brands need to ensure that content shared by users fits their brand.

However, automated detection of sensitive content is complex, requires unique skill sets, and is expensive to build. Human moderation requires significant human effort and does not scale. Amazon Comprehend helps remove many of the challenges for implementing automated detection of profanity and sensitive information such as PII while Amazon Chime SDK messaging provides features that enable integration with Amazon Comprehend to detect profanity, PII, and other undesired content. Amazon Chime SDK messaging also provides features including message editing, message redaction, removing (kicking) users out of channels, and banning users so that they cannot return.

The pattern for processing messages for this demo utilizes streaming export of chat data with Amazon Kinesis. Messages streamed through Amazon Kinesis are then processed by an AWS Lambda function.

message flow

For simplicity, Amazon Comprehend is used for detection of PII and to determine sentiment. For profanity, the Lambda function simply uses a banned word list. This is sufficient for many use cases and does not require training Amazon Comprehend. If you choose, you can also use Amazon Comprehend for profanity, but you will need to train your own model.

Once profanity is detected, a Lambda function is called using a bot with the app instance admin role to remove the content and at first notify the user that they will be kicked and banned if they continue. After the initial warning, this same process is used to again remove the content using the redact API, but this time the bot also removes the user and bans them from the channel.

profanity moderation Profanity Moderation 2

To implement detection of PII, the content of this stream is sent through the AWS Lambda function that processes messages and utilizes Amazon Comprehend to detect PII. Once PII is detected, a Lambda function is called using a bot with the app instance admin role to edit the message, removing the sensitive content.

PII Redaction

Reactions and Sentiment Analysis

In modern chat applications users expect features like reactions, that allow them to react to content in real time. In some cases, this may be a reaction to content in the channel, in other cases it may be a chat participant reacting to content displayed elsewhere, such as broadcast video. Adding reactions that contribute to the shared experience of chat participants requires low-latency messaging that scales.

Sentiment analysis of chat data has multiple uses. In our demo we show a basic example that ties in with the implementation of reactions, but the same concept can be used for different implementations. For a large channel, analysis of chat content could be one metric to determine the overall response to a product demonstration or keynote, with the possibility to identify individual users who were especially positive in their response. Outside of the context of a large chat room, this same type of analysis could be used in a dashboard to identify bad customer interactions such as when a customer is responding poorly to a conversation with a delivery driver.

In this demo, reactions and sentiment analysis follow the flow we have set up for detection of PII. Each message is streamed through the Amazon Kinesis data stream and into the AWS Lambda function, then to Amazon Comprehend. Amazon Comprehend determines the sentiment of the message, which is then sent back to Lambda. From there, the bot sends a control message down to the clients with the sentiment which then triggers the display of the reaction. At the same time, the sentiment score at the top of the page is updated to account for the new message. If the message is positive, the positive score increases. If the message is negative, the negative score increases. If the message is neutral, both scores will drop since we are not accounting for neutral sentiment in the UI.

Sentiment Analysis with Reaction

This is just one implementation of a reaction. Amazon Comprehend again is optional here, if you simply want to detect specific words and react to them rather than sentiment you can include that list of words directly in the AWS Lambda function. This can be enhanced by adding additional types of reactions, counting the number of times a reaction is sent, etc. You can, of course, also choose to implement UI such as buttons to trigger these reactions, or allow users to react to messages in the channel. You as the developer have full control over the experience of your users.

Clean Up

If you don’t want to continue to be charged for the use of the moderated chat demo app, you can clean up by deleting the AWS CloudFormation stack and resources created in the section for deploying the Amazon Chime SDK moderated chat demo app.

To delete the stack and its resources:

  1. From the AWS CloudFormation console in us-east-1, select the stack that you created for this demo.
  2. Click Delete Stack.
  3. In the confirmation message that appears, click Yes, Delete. At this stage, the status for your changes to DELETE_IN_PROGRESS. In the same way you monitored the creation of the stack, monitor its deletion by using the Events tab. When AWS CloudFormation completes the deletion of the stack, it removes the stack from the list.

Conclusion

In this post, you learned how you can use the features of Amazon Chime SDK messaging alongside other AWS services including AWS Lambda and Amazon Comprehend to add auto-moderation to a chat channel that can be used alongside a broadcast video or in any use case where you have a large chat rooms that require moderation or sentiment analysis.

The Amazon Chime SDK messaging features are available today in the US East (N. Virginia) region at low per-message rates. To get started with Amazon Chime SDK messaging, read our developer guide.