US Open AI Commentary with IBM watsonx
The 6th Annual Awards
- Best New Fan Experience
ABOUT THIS ENTRY
IBM built a generative AI solution to add spoken commentary to the video highlight reels for every singles match of the 2023 US Open tennis tournament. Using a large language model (LLM) called Sandstone, available through IBM's watsonx.ai platform, the team built, trained, tuned, and deployed the solution to automate the production of detailed audio narration and captions to accompany the highlight video packages on USOpen.org and the US Open app. The team trained the LLM on extensive tennis data, including trusted USTA data curated through the watsonx.data platform to ensure the data's integrity and filter out erroneous and objectionable content. The team then trained the model to translate the metadata from video clips and turn them into a variety of sentences, which were converted to audio with Watson text-to-speech technology, giving tennis fans a more engaging experience.
How does this represent “Excellence in Engagement”?
AI’s startling rise to the forefront of public consciousness in 2023 stirred up equal parts excitement and anxiety over its role in creating the future. Despite the incredible power demonstrated by public-facing generative AI tools like ChatGPT, its immediate utility for commercial and enterprise applications was uncertain. IBM and the USTA’s application of generative AI for US Open’s highlight commentary successfully demonstrated the capability of foundation models to produce meaningful business outcomes when adequately trained, tuned, and governed. The commentary and captions allowed the USTA to accompany the hundreds of untelevised matches with spoken and written context that was previously impossible given the constraints of human broadcasters, enriching the highlight reels and making them accessible for those with visual and hearing impairments. The success of the project opened the doors of possibility not only for the use of generative AI to elevate the engagement of sports SVOD experiences for the rapidly burgeoning market of online and app-based consumption but also for the use of AI in high-visibility commercial applications in general. Through its use of watsonx’s data curation and governance tools, the project proved that AI can yield transformative results when properly trained and regulated.
Objective
The US Open is a two-week tournament with hundreds of matches played on 22 different courts. Most of those matches are not broadcast on television and, therefore, do not have spoken commentary. To make the highlights from these matches more engaging for the more than 15 million global tennis fans who follow the tournament through the US Open app and website, the United States Tennis Association (USTA) turned to longtime digital experience partner IBM Consulting for a cutting-edge solution. For over three decades, IBM and the USTA have collaborated on innovative digital experiences for the US Open, including AI-based insights into match data. To keep the digital experience fresh and meet the modern demands of the tournament’s vast global audience, the USTA harnessed IBM’s new watsonx generative artificial intelligence (gen AI) suite to create spoken commentary and captions for the match highlights automatically.
Strategy & Execution
Modern sports fans are moving away from traditional broadcast television and increasingly favoring streaming video-on-demand (SVOD) platforms and content. According to a recent Deloitte study, more than half of surveyed fans prefer the SVOD experience over cable or broadcast TV. Against this backdrop, the USTA and IBM set out to enrich the SVOD experience for the US Open and make it more accessible by adding AI commentary and captioning. The first step in empowering the Sandstone LLM in watsonx.ai to produce commentary was to give it domain expertise. This intensive training process enables the model to “learn” all about tennis. Using the watsonx.data platform, the team connected and curated the USTA’s extensive database and guided the process by de-duping and filtering the data used to inform the model. The AI Commentary model is then trained to translate the metadata attached to video clips into sentences. This involves interpreting over 7 million data points, including serve direction, speed, shot type, and ball position for the more than 125,000 points played. The AI then generates dozens of options before choosing the best sentence to describe the action, taking care to vary the sentence structure to avoid repetition.
Organizations
- IBM
- USTA
Links
Credits
Noah Syken
Vice President, Sports and Entertainment Partnerships (Global Sponsorships) at IBM
IBM
Kristi Kolski
Program Director, Sports & Entertainment Partnerships at IBM
IBM
Tyler Sidell
Technical Program Director, Sports & Entertainment Partnerships at IBM
IBM
Corey Shelton
IBM, Technology Program Director, Sports & Entertainment Partnerships
IBM
Celebrate in NYC
Join us for the 6th Annual Hashtag Sports Awards™ on the evening of June 6, 2024 as we celebrate finalists and reveal the winners of this year's awards live during the show.
