Kyndryl’s AWS Certification data explorer using Amazon Q in QuickSight

This is a joint post authored with Bhupesh Gupta, Jigar Kapasi, and Mike Williams from Kyndryl.

Amazon Q in QuickSight uses natural language processing (NLP) to improve how we interact with data and generate insights. It effectively adds generative business intelligence (BI) features to Amazon QuickSight. Amazon Q in QuickSight simplifies data analysis for business users by providing executive summaries, dashboard authoring capabilities using natural language, customizable data stories, and a context-aware data Q&A experience.

Adding NLP capabilities to the analytics domain enhances analyst productivity and provides additional tools to discover meaningful insights. The promise to the business user is to make self-exploration of data using NLP simpler. Quick access to clear and concise data insights supports data-driven business decisions. Additionally, faster analysis and insights can lead to quicker decision-making and data-driven innovation, potentially saving time and resources while helping businesses stay ahead of the curve in competitive markets.

Kyndryl is an AWS Premier Tier Services Partner and Managed Service Provider that is continuously innovating to support customers in their transition to cloud-centered environments. In this post, we demonstrate how Kyndryl used Amazon Q in QuickSight to build a Certification Data Explorer tool for analyzing company-wide employee AWS technical certification data, enabling quick insights and data-driven decision-making.

At Kyndryl, we have an ongoing requirement to analyze company-wide employee AWS technical certification data to draw insights and make better planning and investment decisions. For this use case, we chose Amazon Q in QuickSight to build a Certification Data Explorer tool to quickly gain various perspectives and make data-driven training and enablement decisions. This solution is used by senior leadership, who are given Reader Pro access in Amazon Q in QuickSight. Users in Kyndryl’s AWS landing zone authenticate themselves using Kyndryl’s enterprise single sign-on (SSO) integration into AWS IAM Identity Center.

The base Kyndryl employee certifications data is available through the AWS Partner portal as a CSV file. The data was further enriched with additional information that’s meaningful in Kyndryl’s context. We examined thousands of AWS Certification records, which had several attributes such as certification name, date, type, expiration date, candidate email, employee details, and so on. The data seen in this post has been altered to protect personal and corporate information.

Preparing for analysis with Amazon Q in QuickSight

A few setup steps are required before we can use natural language to generate insights and dashboards from the data.

Prepare the dataset

The first step is to make the dataset available in QuickSight. The source data for this example is in CSV format. To create a new dataset, complete the following steps:

1. On the QuickSight console, choose Datasets in the navigation pane.
2. Choose New dataset, then choose Upload a file.
3. While creating the dataset, you might have to provide the format of certain fields, such as Date fields. You can provide the desired format of the field by choosing Edit settings and prepare data.

Alternatively, you can use the Edit option in the datasets listing (for an existing dataset). In our use case, we needed to set the format of two date fields, as shown in the following screenshot.

Create calculated fields using natural language

In some scenarios, you might need a new data field derived from one or more existing fields. The calculated fields editor in QuickSight helps you add such fields. The editor provides a syntax and a set of statistical and mathematical functions to write expressions.

Amazon Q in QuickSight creates expressions in response to natural language prompts. You can review and edit the expressions before adding them to calculations. Complete the following steps:

1. On the QuickSight console, choose Analyses in the navigation pane.
2. Select the desired analyses, and on the Data menu, choose Add calculated field with the Build calculation option.

Using natural language to generate expressions significantly reduces the amount of time it takes for those new to QuickSight or unfamiliar with specific expression syntax to complete tasks.

Create Amazon Q topics

Amazon Q in QuickSight capabilities are accessed by creating an Amazon Q topic in QuickSight.

1. On the QuickSight console, choose Topics in the navigation pane.

2. Enter a topic name and description.
3. Select Use new generative Q&A experience.

You won’t be able to ask questions in natural language if this check box isn’t selected.

4. Select the dataset (AWS Certification data in our use case) to be analyzed by QuickSight and choose Create.

You will see a status window showing the progress of the analysis, as shown in the following screenshot.

After topic creation, the process of data field selection, classification, and creation of new data fields begins. QuickSight provides automated field selection to accelerate the process, as shown in the following screenshot. However, you should validate the automated field and modify them if necessary.

You also might need to define and validate field attributes. This is done automatically by Amazon Q in QuickSight, and you can further refine the attributes manually on the QuickSight console. For instance, as shown in the following screenshot, we changed the Date field’s role from Dimension to Measure. This review needs to be done for each active field in the analysis. You can learn more about dimensions and measures and how to set and use dimension and measure fields.

Finally, automated data prep by Amazon Q in QuickSight continues by automatically labeling fields and identifying synonyms. Again, QuickSight provides the ability to manually override the automatic suggestions if needed. In our case, our dataset had a Country column that indicated an employee’s home country. Amazon Q suggested using the synonym Nation, which isn’t commonly used at Kyndryl. We provided a more acceptable alternative of Geography instead. In another instance, the Certification Name column that stored the AWS Certification title, was interpreted by Amazon Q as Course Title and Qualification. Because these terms aren’t generally used in the AWS Certification context, we changed it to the more commonly used Certification.

See Preparing data in Amazon QuickSight for additional information. With our data in good shape, we can move on to working with Amazon Q in QuickSight for our use case.

Analyzing data with Amazon Q in QuickSight

To access QuickSight generative BI, choose the Q icon at the top right of your QuickSight page. A question prompt text box along with some suggested questions will appear in the pane, as shown in the following screenshot. If you have more than one topic, make sure the correct topic is selected from the dropdown menu on the top left. You can review and delete the automatic suggestions as needed. This can be done by choosing the Suggested Questions tab on the Topic page. You can also promote questions by verifying them.

You can use several techniques to improve the performance of the Amazon Q in QuickSight application. These include things like prompt engineering, asking and adjusting questions, modifying the Amazon Q interpretation of the question, giving feedback on the responses, and promoting good questions. Amazon Q in QuickSight suggests field names or synonyms as defined in the dataset, as you input your questions, which results in more meaningful prompts for the model. The following screenshot shows the response to a user’s question in Amazon Q.

As you ask questions and generate various graphs, they can be pinned to the pinboard for later reference. Graphs are also generated on the Analyses dashboard using generative AI features, which are used while generating data stories.

Data stories and dashboard capabilities

Among other features, you can generate an executive summary from a dashboard and build powerful data stories. A snapshot of the Amazon Q in QuickSight dashboard for our AWS Certification data explorer use case is shown in the following screenshot. All visuals in this dashboard were generated using natural language prompts with the Amazon Q Build visual capability in QuickSight analyses, as we demonstrated previously.

We used the data stories generative AI feature to create reports and articles. For example, we created a detailed report of analysis and recommendations for senior management based on a prompt and visuals created earlier. The following screenshots capture the Build story UI with the prompt given and a snapshot of the resulting report.

The following screenshot shows the resulting report. We observed that Amazon Q interpreted the prompt well and created a report with meaningful insights that can be readily used with only a few minor updates.

Conclusion

In this post, we showed how we used Amazon Q in QuickSight to accelerate a certification data analysis use case. In addition to offering a generative BI experience, the service offers numerous features, including user control, publishing, sharing, subscribing to reports, embedded analytics, data protection, and access management, which are essential for enterprises. With its emphasis on generative AI and enterprise controls, Amazon Q in QuickSight can be a powerful BI tool, empowering organizations to make informed decisions faster.

To explore QuickSight and how its cutting-edge features can deliver new applications, reduce in-house development time, and make data insights faster, smarter, and more accessible, see Amazon QuickSight.

About the Authors

Bhupesh Gupta is a Cloud Solutions Architect in the AWS CCoE in Kyndryl’s CTO organization. He provides AWS and public cloud solutions and first-of-its-kind implementations to the clients. In his spare time, Bhupesh enjoys doing DIY projects and following politics, economy, and commerce.

Jigar Kapasi is a Senior Architect in the AWS CCoE within Kyndryl’s CTO organization. He leads a team of seasoned experts who develop and review reference architectures, explore emerging technologies, and nurture an AWS technical community. In his spare time, Jigar enjoys following Indian cricket, reading, practicing yoga, and cycling.

Mike Williams is a Kyndryl Fellow and VP in the Kyndryl CTO organization. He leads the technical strategy, architecture, and design of the next generation of services supporting our client’s AWS Cloud journey. In his spare time, Mike enjoys cycling, hiking, and skiing with his family and friends.

Asif Fouzi is a global technology leader at AWS supporting Global Service Integrators (GSI), helping GSIs such as Kyndryl in their cloud journey. When he is not innovating on behalf of customers, he likes to play guitar, travel, and spend time with his family.