Engage online sports fans with live event commentary using generative AI on Amazon Bedrock

Live sports online streaming is a fast-growing market, forecast to grow from USD $18.6B in 2021 to USD $93.1B by 2027, a compound annual growth rate of 24.64% from 2022 to 2030. Sports streaming platforms continually advance, employing diverse strategies to delight users and enrich viewer engagement. These tactics encompass extended statistics and match facts, augmented reality experience, predictive analytics, and live in-race commentaries.

Providing dynamic narration and commentary during sports events serves to captivate the audience, spur friendly competition amongst fans, and create an immersive race-day atmosphere. Live commentaries generated during a sports event are typically a blend of human expertise and automated processes. Trained commentators deliver real-time insights, while speech-to-text technology transcribes spoken words for analysis. Data integration, including live statistics feeds and match facts, adds context. Operators may also incorporate fan-generated content for community engagement. Recently, the integration of AI-driven generative models became common. Through its ability to generate relevant textual content from structured data inputs, generative AI can be a powerful tool to automate commentary generation, by streamlining content creation based on live facts from the sports event.

In this blog post, we explain how to create a live sports commentary stream that automatically adapts to the action. By ingesting real-time game statistics and facts, we leverage AI generative models to output contextually relevant text and commentary that is tailored to the ongoing sports event. We use Amazon Web Services (AWS) serverless technology, including Amazon Bedrock to invoke Large Language Models (LLM) for content generation, Amazon DynamoDB tables to store game statistics and facts, and Amazon Lambda integration with API Gateway to expose APIs towards the operator’s online streaming application.

Prerequisites

Live statistics and match facts are key drivers for generating relevant commentaries during a sports event. While these are treated as input data in this post, we refer the reader to Behind the scenes: AWS real-time architecture for Bundesliga Match Facts to learn more about an example solution for how to build a similar pipeline for live match facts and statistics generation using managed AWS services. This solution makes use of Amazon MSK to distribute events in near real time, and supports integration with Amazon DynamoDB to output statistics data for storage. An existing DynamoDB table, referred to as a statistics feed table, is a prerequisite for implementing the solution described in this post. It is assumed to be already present in the user account, and updated with a live stream of raw data and statistics from the ongoing sports event.

Overview of the solution

Assuming the user already has an existing pipeline to deliver live statistics and match facts for its sports events on AWS, as previously outlined in the prerequisites section, the architecture that we describe in this post is illustrated in the following figure.

Figure 1: High level architecture diagram of the In-race commentaries generation solution

This workflow includes five steps, described as follows:

• First, the operator’s online streaming application streams live video content from the sports event. Commentaries are generated live during the event and displayed to viewers as an updated feed of in-race commentaries within the application interface.

• The Amazon API Gateway WebSocket API enables bidirectional communication where the client streaming application can request new live commentaries from the service, and the service can independently push new data feeds to the application in near real time.

• The in-race commentary generation service is implemented as an event-driven serverless application using AWS Lambda integration with API Gateway. The Lambda function leverages two DynamoDB tables:

• • DynamoDB-Backed Chat Memory serves longer-term persistence of session memory data.
  • The statistics feed table provides access to ongoing sports’ event raw analytics data.

• The Lambda function generates race commentaries live during the event by invoking the Amazon Bedrock service API. It first appends the latest statistics feed data to the session memory store, and then invokes the LLM to generate personalized in-race commentary based on the updated session data now available.

• Finally, the Lambda function returns the new in-race commentary feed to the operator’s online streaming application through the open WebSocket API.

Commenting a live horse race event

To illustrate the solution described in this post, we use horse races as an example scenario. Horse races are fast-paced events with frequent position changes and events like overtakes that lend themselves well to real-time commentary. While the details focus on horse racing, the automated commentary approach we outline could be applied to other sports and competitive events as well.

We first explain the data structure that we use to capture the race facts and statistics throughout a horse race event. Then, we introduce the prompt engineering phase and how we invoke the LLM model to create in-race commentaries. Finally, we explain how we streamline the process to generate in-race commentaries live during the sport event.

How we model the race statistics feed

To capture live statistics from a horse race, we use in this post a simple data template to illustrate the use case. The template includes timestamped data points from the race start till the end, as illustrated in the following example data point.

{
        "Item": {
                “race-id”: {“S”: “12345”},
                "timestamp":{"S":"1"},
                "race_finished":{"BOOL":false},
                "number_competing_horses":{"N":"2"},
                "competing_horses":{"L":[
                                {"M":{
                                        "horse":{"S":"Orange"},
                                        "ranking":{"N":"1"},
                                        "jockey":{"S":"John Doe"},
                                        "speed":{"N":"30"}
                                        }
                                },{"M":{
                                        "horse":{"S":"Magenta"},
                                        "ranking":{"N":"2"},
                                        "jockey":{"S":"Jessy Doe"},
                                        "speed":{"N":"28"}
                                        }
                                }]
                },
                "weather":{"S":"rainy"}
        }
}

This template feeds in metrics like the participating horse and jockey names, their current placement and speed at each moment, any lead changes, and the elapsed time since the race has started.

The timestamp allows us to anchor each update to the progress of the event, while the other metrics capture details that are needed to provide an engaging race day experience for fans. Setting a number_competing_horses parameter in each data feed establishes how many participants are tracked, to leverage cases where some horses may have crossed the finish line or, rather, have abandoned the race. Additionally, each data feed contains a race_finished boolean that defaults to False, and is switched to True once all the horses have crossed the finish line to indicate the conclusion of the event.

Designing the prompts for LLM invocation

The prompt engineering is how we guide the LLM to generate content that is tailored to the context of a real-time commentary feed for a live horse race event. To illustrate the example in this post, we use Anthropic’s Claude, which is available on Amazon Bedrock, and we use the following prompt template to set the context for LLM for an online live media commentator.

PROMPT = “““ Human: You are an online live media commentator, commenting events as they unfold for an online horse race event. You create live comments during the race, based on a live statistics feed about the horse race event, delivered as contextual data.

Current conversation:
<conversation_history>
{history}
</conversation_history>

Here is the human's next reply:
<human_reply>
{input}
</human_reply>

Assistant:
"""

• Commenting the competition before the starting gate opens: Before the race starts, the prompt guides the LLM to generate a commentary to set the stage, highlighting key data points about the competing horses and jockeys, and to establish an informative narrative leading into the running.

We use the following prompt template:

RACE_START_INPUT = "As the horse race starts in a moment, given the facts about the event that are specified in the session context, write a comment that sets the stage for the race and engage the fans for a new race that is about to start."

• Commenting in-race events as the horses round the track: As the race unfolds, the prompt guides the LLM to generate dynamic play-by-play commentary, reacting to live data feeds to emphasize lead changes and other pivotal moments of the race.

We use the following prompt template:

INRACE_INPUT = "The race is ongoing, as you are provided with updated race facts, write an in-race commentary to describe updates in racing positions and horse rankings. Comments should be engaging for the fans and should reflect the race momentum."

• Recapping the race as the final results unfold: By the end of the race, the prompt guides the LLM to produce an analytical summarization, by recounting the race in short and commenting the results.

We use the following prompt template:

ENDRACE_INPUT = “The race has now ended. The last race facts have now been provided. Write a commentary to summarize the key facts during the race, reflecting also on the final race results and providing concluding remarks to the audience.”

Streamlining in-race live commentaries

Streamlining commentaries generation is the part where we assemble the pieces together to deliver a live feed to the online media platform. This includes the process by which we combine live statistics and match facts from the race with the prompts to invoke the LLM model. We use LangChain open-source framework that provides modules to integrate with the LLM of choice, and orchestration tools for activities such as chat history management and prompt engineering.

To sustain a complex feed of commentaries, where each new comment expands previously generated comments and leverages the most recent race facts, we use the LangChain memory system, which offers integration with DynamoDB to store message history. The following code snippet explains how to set up a conversation chain with DynamoDB to manage session memory, and Anthropic Claude on Amazon Bedrock for comment generation:

import boto3
from langchain.chains import ConversationChain
from langchain.llms.bedrock import Bedrock
from langchain.memory import ConversationBufferMemory
from langchain.memory.chat_message_histories import DynamoDBChatMessageHistory

SESSION_TABLE = < Name of DynamoDB session table >
RACE_ID = <Unique Identifier for a horse race event >

FeedHistory = DynamoDBChatMessageHistory(table_name=SESSION_TABLE, session_id=RACE_ID)
memory = ConversationBufferMemory(memory_key="history", chat_memory=FeedHistory, return_messages=True,ai_prefix="A",human_prefix="H")
CommentGen_llm = Bedrock(model_id = "anthropic.claude-v2", model_kwargs = inference_modifier )
conversation = ConversationChain(llm=CommentGen_llm, verbose=False, memory=memory)

Next, populate the session memory with updated facts and race statistics stored in the statistics feed table. The following code snippet extracts the latest feeds for the current horse race from the statistics table, and loads them into the conversation chain memory.

FACTS_TABLE = < Name of DynamoDB statistics feed table >
MaxFeeds = < Maximum number of new feeds >

def scan_lastest_feeds (table_name, race_id, MaxFeeds):
    dynamodb = boto3.resource('dynamodb')
    table = dynamodb.Table(table_name)
    response = table.scan( Limit=k, ScanIndexForward=False)
    items = [item for item in response['Items'] if item.get('race-id') == race_id]
    return items

newFeeds = scan_lastest_feeds (FACTS_TABLE, RACE_ID, MaxFeeds)
if newFeeds :
    FeedHistory.add_user_message(str(newFeeds))

Scanning the latest data from the statistics feed table, and adding them to the conversation chain memory, allows invocation of the LLM while using the latest race statistics as we generate in-race commentaries.

The final step involves orchestrating LLM invocations during the live sports event to provide a continuous stream of in-race commentaries. This functionality is achieved through a lambda function, which handles API requests from the online streaming application via the WebSocket API. When triggered, the lambda function utilizes the ‘scan_latest_feeds’ function to retrieve the most recent updates from the statistics feeds, integrates them into the conversation chain memory, and triggers the LLM to generate a new commentary. This commentary could be an opening statement if the race hasn’t begun yet, an ongoing commentary during the event, or a summary commentary at the conclusion of the race.

Following is a code snippet demonstrating this process.

from langchain.prompts import PromptTemplate

def generate_commentary(table_name, race_id, MaxFeeds):
newFeeds = scan_lastest_feeds (table_name, race_id, MaxFeeds)
if newFeeds: 
    FeedHistory.add_user_message(str(newFeeds)) 
    conversation.prompt = PROMPT
    if (newFeeds[-1].get(‘timestamp’) == ‘0’): # Race not started yet
       return conversation.run(input=RACE_START_INPUT)
    elif newFeeds[-1].get(‘race_finished’): # Race has ended
       return conversation.run(input=ENDRACE_INPUT)
    else: # Race is still ongoing
       return conversation.run(input=INRACE_INPUT)

Example

The following screenshot displays a web interface simulating a media streaming portal to illustrate the output produced by the solution in this post. The right panel shows simulated live match facts and statistics being added to the statistics feed table. The left sidebar contains automatically generated in-race commentaries created by the solution based on the live statistics feed. This demonstrates how the architecture ingests real-time data to generate contextual narrative commentary for sports events using generative AI.

Figure 2: Example that illustrates the solution in action during a horse race event

Conclusion

In this post, we described a solution that generates engaging, real-time commentary for live sports events using generative AI. By integrating Amazon Bedrock to invoke large language models with a serverless architecture on AWS, we created an automated pipeline that ingests live statistics and facts to produce contextual narrative tailored to the event’s ongoing action. This allows sports media platforms to cost-effectively scale personalized, dynamic commentary feeds that captivate fans. While demonstrated for the example of a horse racing event, the principles apply across sports verticals. Overall, this post showcased how AWS serverless building blocks can be combined with the unique capabilities of generative AI, available on Amazon Bedrock, to transform viewer engagement for the growing online live sports streaming industry.