BT’s ‘GenAI Gateway’ platform powered by AWS Launched

A single consolidated Group platform reduces duplication of effort and resources as BT Group scales and accelerates the adoption of generative AI as APIs, security configuration, infrastructure management etc., can be managed centrally, reducing the risk of error along the way.

BT Group’s Digital Unit has announced the launch of an innovative internal platform to help the company tap into the power of large language models (LLMs) from providers such as Anthropic, Meta, Claude, Cohere, and Amazon. The GenAI Gateway, built in collaboration with AWS and using Amazon Bedrock, Amazon SageMaker and AWS Professional Services capabilities, provides secure, private access to a range of natural-language processing and large language models, a critical tool BT Group will use as it embeds AI into the way it runs the business.

Using LLMs on an ad-hoc basis is fine for testing and development, but it’s not ideal for large-scale use. This is because managing costs, security, and privacy requires more attention. Additionally, LLM performance must be monitored for unexpected errors, like "hallucinations," and potential model decay, where the LLM stops working as expected over time. The GenAI Gateway helps BT Group avoid being locked into a single LLM if other issues arise. This platform also encourages BT Group engineers to select the best model for each specific use case, while tracking budgets for each use.

Deepika Adusumilli, Managing Director, Data & AI, BT Group’s Digital Unit said: “AI is helping us reimagine the future of our company. We believe that where our data is a constant, we need flexibility with our LLMs. GenAI Gateway allows us to tap into this powerful new set of technologies at scale, in a way that is safe, responsible, flexible and scalable, delivering the ambition we have for AI to unlock the human potential within BT Group, today and in the future.”

GenAI Gateway, deployed on AWS, is accessed via secure APIs, like all the components of BT Group’s modular digital architecture. GenAI Gateway uses Amazon Bedrock, a fully managed service that offers a choice of high-performing foundation models from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API; as well Amazon SageMaker, a fully managed service that brings together a broad set of tools to enable high-performance, low-cost machine learning for any use case.  

Fabio Cerone, GM EMEA Telco at AWS, shared, “The BT Group GenAI Gateway is showcasing how enterprises can effectively deploy generative AI at scale and speed. It’s been a brilliant, pioneering opportunity to collaborate and work backwards from the customer to provide a way to accelerate deployment of generative AI use cases into production with embedded security and compliance. The GenAI Gateway will trigger the flywheel effect in the adoption of generative AI, delivering quicker results for BT Group and its customers.”

The platform supports prompt security, chat history, FinOps billing per use case, enterprise search, as well as use of multiple corporate data sources. Central privacy controls include separate tenants for each use case, the use of Personal Identifiable Information filters, the location of the data within the UK and the isolation of trained models from each other, protecting data in line with Group policies and relevant regulation.

Gen AI Gateway is one of the several key enablers we are deploying to enable BT Group as an AI-enabled enterprise, and we will also use our “data fabric” data management platform to help enforce governing policies for how data can be used, as well as to manage access control and data sovereignty restrictions.