AWS Contact Center

Build a drug reminder service with AWS IoT, Amazon Lex, and Amazon Connect

Taking medications as prescribed is essential for any illness treatment. However, it can become confusing when there are multiple pills to be taken at different times of the day. This becomes more problematic for patients with mild cognitive impairment, Alzheimer’s or other kinds of dementia. Some medications are prescribed three times a day, while others are taken on some days and not on others. In this case, a drug reminder system would be helpful.

Healthcare institutes can build a service to remind their patients to take drug dosages on time. Building an effective drug adherence program not only improves the health of the patient suffering from chronic illnesses but also saves health costs.

In this blog, you will learn how to build a virtual healthcare assistant that makes outbound reminder calls to patients that forget to take their medicine at the prescribed time. The virtual healthcare assistant solution uses AWS IoT to monitor the medicine box, Amazon Connect to manage calls, Amazon Lex as the conversational agent, and AWS Lambda to manage the data and connect to the backend patient database (CRM).

Overview of solution

This post provides an example of how to build a reminder service for Alzheimer’s patients. We pull the records of the customers who need drug reminders using AWS Lambda. AWS IoT will monitor those patients’ medicine box at the prescribed time using sensors and will prompt Amazon Connect to make an outbound call to the patient, if they miss their dose. Amazon Lex will help them take the correct medicine and answer questions they may have.

The sample below illustrates the interaction between bot and Alzheimer patient.

Bot: Hi, I am your virtual healthcare assistant. I am calling you about your afternoon medication. Have you taken your medication yet?

Patient: Oh, I’m not sure. What does it look like?

Bot: It is a medium, round, pink tablet, and it comes in a white box.

Patient: How many do I need to take?

Bot: You need to take one tablet now. Let me know once you’ve taken it.

Patient: Okay, I took it.

Bot: Okay. Is there anything else I can help you with?

Patient: No.

Bot: Thanks. Have a great day.

Architecture diagram

The following diagram represents the solution.

Workflow Sequence:

1.      Salesforce admin creates patient records
2.      Fetch patient data for reminders
3.      Insert patient data
4.      Send box status every 5 seconds and trigger AWS Lambda
5.      Compare reminder time with current time (User Time zone). If reminder time is within 5 minutes window, update calling flag to True, if the box is closed
6.      Update dosage has been missed
7.      Make outbound call and transfer it to Amazon Lex bot
8.      Update the outcome of the call to CRM

Prerequisites

For this walk through, you should have the following prerequisites:

If you are using this data in production, enable access logging and follow the best practices mentioned in the link below.

https://d1.awsstatic.com/whitepapers/compliance/AWS_HIPAA_Compliance_Whitepaper.pdf

Walkthrough
Summary of the steps:

  •   Create a customer record in Salesforce
  •   Create an AWS IoT device
  •   Install and configure the Arduino IDE
  •   Configure the ESP32 IoT device
  •   Flash the ESP32
  •   Deploy the AWS CDK application
  •   Validating the solution

Step-by-step instructions
Create a customer record in Salesforce

Salesforce Admin creates a customer record with the following fields and values. Additional information on how to create custom fields within Salesforce can be found here.

  1. Customer_Phone_Number: +1XXXXXXXXXX
  2.  Time_Dosage: 2021-09-07T19:50:00.000+000
  3. Medicine_Color: Pink
  4.  Medicine_Shape: Round
  5. Medicine_Size: Medium
  6.  Medicine_Location: White box
  7. Reminder_Time: 2021-09-07T19:45:00.000+0000
  8.  Reminder_Required: Yes
  9. Calling_flag = true or false

It’s worth noting that the Salesforce naming conventions for custom fields are identified by a suffix of two underscores immediately followed be a lowercase “c” character as stated here. As an example, to read or use the Reminder_Required field in a query, it would need to be addressed as Reminder_Required__c

Create an AWS IoT device

  1. In the AWS IoT console, choose Register a new thing, Create a single thing.
  2. Name the new thing IoTMedBox. Leave the remaining fields set to their defaults. Choose Next.
  3. Choose Create certificate. Only the thing cert, private key, and Amazon Root CA 1 downloads are necessary for the ESP32 to connect. Download and save them in a secure place. They will be used when programming the ESP32 device.
  4. Choose Activate, Attach a policy.
  5. Skip adding a policy, and choose Register Thing.
  6. In the AWS IoT console side menu, choose Secure, Policies, Create a policy.
  7. Name the policy IoTMedBoxPolicy. Choose the Advanced tab.
  8. Paste in the following policy template : { "Version": "2012-10-17",
    "Statement": [
    { "Effect": "Allow", "Action": "iot:Connect", "Resource": "arn:aws:iot:REGION:ACCOUNT_ID:client/IoTMedBox" },
    { "Effect": "Allow", "Action": "iot:Subscribe", "Resource": "arn:aws:iot:REGION:ACCOUNT_ID:topicfilter/esp32/sub" },
    { "Effect": "Allow", "Action": "iot:Receive", "Resource": "arn:aws:iot:REGION:ACCOUNT_ID:topic/esp32/sub" },
    { "Effect": "Allow", "Action": "iot:Publish", "Resource": "arn:aws:iot:REGION:ACCOUNT_ID:topic/esp32/pub" }
    ]
    }
  9.   Replace REGION with the matching AWS Region you’re currently operating in. This can be found on the top-right corner of the AWS Management Console window.
  10. Replace ACCOUNT_ID with your own, which can be found in Account Settings.
  11. Choose Create.
  12. In the AWS IoT console, choose Secure, Certification. Select the one created for your device and choose Actions, Attach policy.
  13. Choose IoTMedBoxPolicy, and Attach.

Your AWS IoT device is now configured to have permission to connect to AWS IoT Core. In addition, it can publish to the topic esp32/pub and subscribe to the topic esp32/sub. For more information on securing devices, see AWS IoT Policies.

Install and configure the Arduino IDE

The Arduino IDE is an open-source development environment for programming microcontrollers. It supports a continuously growing number of platforms, including most ESP32-based modules. It must be installed along with the ESP32 board definitions, MQTT library, and ArduinoJson library.

  1. Download the Arduino installer for the desired operating system.
  2. Start Arduino and open the Preferences window.
  3. For Additional Board Manager URLs, add
    https://dl.espressif.com/dl/package_esp32_index.json
  4. Choose Tools, Board, Boards Manager.
  5. Search esp32 and install the latest version.
  6. Choose Sketch, Include Library, Manage Libraries.
  7. Search MQTT, and install the latest version by Joel Gaehwiler.
  8. Repeat the library installation process for ArduinoJson and ESPDateTime

The Arduino IDE is now installed and configured with all the board definitions and libraries needed for this walkthrough.

Light sensor setup with ESP32

This is a sample setup of ESP32 board with the Digital Light Intensity Detection Photosensitive Sensor Module.

Hardware used for the setup:

You can also run this setup and many other sample use cases on the Core2 for AWS IoT Edukit.

Configure the ESP32 IoT device

For this section, you need an ESP32 device. To check, if your board is compatible with the Arduino IDE, see the boards.txt file. This project has been tested on M5Stack Core / M5Stick.

The following code connects to AWS IoT Core securely with MQTT, a publish and subscribe messaging protocol.

  1. Install the required serial drivers for your device. Some boards use different USB/FTDI chips for interfacing. CP2104 Driver is the most commonly used.
  2. Open the Arduino IDE and choose File, New to create a new sketch.
  3. Add a new tab and name it secrets.h
  4. Paste the following into the secrets file.
  5. #include <pgmspace.h>
#define SECRET
#define THINGNAME ""
const char WIFI_SSID[] = "";
const char WIFI_PASSWORD[] = "";
const char AWS_IOT_ENDPOINT[] = "xxxxx.amazonaws.com";
// Amazon Root CA 1
static const char AWS_CERT_CA[] PROGMEM = R"EOF(
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
)EOF";
// Device Certificate
static const char AWS_CERT_CRT[] PROGMEM = R"KEY(
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
)KEY";
// Device Private Key
static const char AWS_CERT_PRIVATE[] PROGMEM = R"KEY(
-----BEGIN RSA PRIVATE KEY-----
-----END RSA PRIVATE KEY-----
)KEY";
  6.   Enter the name of your AWS IoT thing, IoTMedBox, in the field THINGNAME.
  7. To connect to Wi-Fi, add the SSID and PASSWORD of the desired network. Note: The network name should not include spaces or special characters.
  8. The AWS_IOT_ENDPOINT is located in the Settings page of the AWS IoT console.
  9. Copy the Amazon Root CA 1, Device Certificate, and Device Private Key to their respective locations in the secrets.h file.
  10.  Choose the tab for the main sketch file and paste the following.
#include "secrets.h"
#include <WiFiClientSecure.h>
#include <MQTTClient.h>
#include <ArduinoJson.h>
#include "WiFi.h"
#include "ESPDateTime.h"

// The MQTT topics that this device should publish/subscribe
#define AWS_IOT_PUBLISH_TOPIC   "esp32/pub"
#define AWS_IOT_SUBSCRIBE_TOPIC "esp32/sub"

WiFiClientSecure net = WiFiClientSecure();
MQTTClient client = MQTTClient(256);

void connectAWS()
{
WiFi.mode(WIFI_STA);
WiFi.begin(WIFI_SSID, WIFI_PASSWORD);

Serial.println("Connecting to Wi-Fi");

while (WiFi.status() != WL_CONNECTED){
delay(500);
Serial.print(".");
}

// Configure WiFiClientSecure to use the AWS IoT device credentials
net.setCACert(AWS_CERT_CA);
net.setCertificate(AWS_CERT_CRT);
net.setPrivateKey(AWS_CERT_PRIVATE);

// Connect to the MQTT broker on the AWS endpoint we defined earlier
client.begin(AWS_IOT_ENDPOINT, 8883, net);

// Create a message handler
client.onMessage(messageHandler);

Serial.print("Connecting to AWS IoT");

while (!client.connect(THINGNAME)) {
Serial.print(".");
delay(100);
}

if(!client.connected()){
Serial.println("AWS IoT Timeout!");
return;
}

// Subscribe to a topic
client.subscribe(AWS_IOT_SUBSCRIBE_TOPIC);

Serial.println("AWS IoT Connected!");
}

void publishMessage()
{
StaticJsonDocument<200> doc;
DateTime.begin();
doc["time"] = DateTime.toISOString().c_str();
doc["box_close"] = digitalRead(26);
doc["Customer_Phone_Number"] = <Customer phone number with country code>;
// Example: +1234567890
char jsonBuffer[512];
serializeJson(doc, jsonBuffer); // print to client

client.publish(AWS_IOT_PUBLISH_TOPIC, jsonBuffer);
}

void messageHandler(String &topic, String &payload) {
Serial.println("incoming: " + topic + " - " + payload);

//  StaticJsonDocument<200> doc;
//  deserializeJson(doc, payload);
//  const char* message = doc["message"];
}

void setup() {
Serial.begin(9600);
pinMode(26, INPUT);
connectAWS();
}

uint16_t digitalRead_value = 0;

void loop() {
publishMessage();
digitalRead_value = digitalRead(26);
client.loop();
Serial.println("Pin Digital Values");
Serial.println(digitalRead_value);
delay(60000);
}

10.  In the above code, enter the data for:

a.       Customer_Phone_Number [Note: there is a validation set in place to identify 15 digits or fewer after + sign]

11.  Choose File, Save, and give your project a name.

Flash the ESP32

  • Plug the ESP32 board into a USB port on the computer running the Arduino IDE.
  • Choose Tools, Board, and then select the matching type of ESP32 module. Here, a M5Stack Core / M5Stick was used.
  • Choose Tools, Port, and then select the matching port for your device.
  • Choose Upload. Arduino reads Done uploading when the upload is successful.
  • Choose the magnifying lens icon to open the Serial Monitor. Set the baud rate to 9600.

Keep the Serial Monitor open. Once connected to the Wi-Fi messages received on the topic esp32/sub are logged to this console and the device can publish to the esp32/pub topic. Remember to add permissions to the device policy, if you change or add topics at the top of the sketch.

Note: The code used above is a modified version of the code used in this blog. Refer to this blog if you would like to set up an ESP32 board from a different vendor.

Deploy the AWS CDK application

The resources required for this demo are packaged as an AWS CDK app. Before proceeding, confirm you have CLI access to the AWS account where you would like to deploy your solution.

  1. Open a terminal window and clone the GitHub repository (GitHub Code) in a directory of your choice
  2. Navigate to the cdk-app directory and follow the deployment instructions. The default Region is usually us-east-1. If you would like to deploy to another Region, change the AWS_DEFAULT_REGION similar to the following:

export AWS_DEFAULT_REGION=eu-central-1

Fill in all the required parameters in the env.sh file shown below and save it.

export CONNECT_INSTANCE_ID="<ConnectInstanceID>"
export PROJECT_PREFIX="<ProjectPrefix>"
export TIME_ZONE="US/Eastern"
export CONSUMER_KEY="<SalesForceConsumerKey>"
export CONSUMER_SECRET="<SalesForceConsumerSecret>"
export USER_NAME="<SalesForceUserName>"
export PASSWORD="<SalesForcePassword>"
export SECURITY_TOKEN="<SalesForceUserSecurityToken>"
export ENDPOINT="https://login.salesforce.com"

  •   connectInstanceID: Connect Instance ID is located in the Amazon Connect Console.   Example: 5dc019xx-33xx-42xx-b3xx-xxxxxx
  •   timeZone: User time zone. Example: timeZone=”US/Eastern” . You can find the list of valid time zones here

Now populate (source) the parameters stored in the env.sh file and deploy the cdk stack.

source env.sh
cdk deploy

3.      After the Stack is deployed, update the following AWS Lambda:
Function Name: xxxxCustomerCallingLambdaxxxx

a.       AWS CDK deployment creates a contact flow called drug_reminder_flow. Copy the Flow-ID and update the Lambda environment variable FLOW_ID

b.      update the Lambda environment variable QueueId with any queue IDs [For example: BasicQueue is available on Amazon Connect]  present in the Amazon Connect Instance.

Validating the solution on the console:

  •  Post the step Flash the ESP32, you can verify if the data is being sent to AWS IoT Core, by subscribing to the topic esp32/pub
  •  Test the Lambda function xxxxIoTUpdatePatientTablexxxx by using the payload sent by the above topic

As described in the preceding architecture, the sequence comprises the following steps:

Steps performed by Lambda function:

  1.  xxxxSFDCFetchLambdaxxx – Reads the Salesforce credentials from AWS Secret Manager and queries Salesforce using Calling_flag with value set to Yes.
  2. xxxxSFDCUpdateLambdaxxxx – Inserts customer record in DynamoDB
  3.  xxxxIoTUpdatePatientTablexxxx – Updates the calling flag in DynamoDB, if the box is closed during reminder time set.
  4.  xxxxCustomerCallingLambdaxxxx – Fetches customer record from DynamoDB and makes an API call to Amazon Connect Outbound Calling API, if the customer missed their dosage.

Cleaning up

To avoid incurring future charges, remove all created resources by deleting the AWS CloudFormation.

Conclusion

In this blog post, we showed you how to build a drug reminder service using Amazon Connect, AWS IoT and Amazon Lex. This solution acts as a blueprint to build other IoT and communication solutions. For example, IoT water sensor can detect water leakage and trigger an outbound call to your cellphone informing you about the leak. We look forward to seeing what other solutions you build using this architecture. To get started, visit our GitHub repository and deploy the project!

Should you need help with adding Amazon Connect any of these capabilities to contact flows, reach out to one of many Amazon Connect partners available worldwide.