Azure OpenAI Service Breakdown: GPT-3.5, GPT-4, Use Cases, Users, and Trends
OpenAI Service product manager shares what 18,000 businesses are doing
“RAG [for knowledge management bots] is essentially 80% of use cases that we see on OpenAI today. We are now announcing a new capability in Azure OpenAI Service, which allows organizations to use Azure OpenAI models on their own data. With just a few clicks,…this means businesses can create applications that base responses on their own organizational data with natural language processing.” - Mona Whalin, Microsoft Azure
Mona Whalin, product manager for Azure OpenAI Service, provided a deep dive into the use of OpenAI’s foundation models by over 18,000 businesses. While most everyone may start with OpenAI going directly to the company’s APIs, I am increasingly hearing that they generally shift over to Azure when it’s time to promote their solutions to production. The infrastructure and APIs are more stable and offer elastic scaling. This is even causing some companies to try Azure for the first time.
Generative AI Use Cases
Whalin took the stage at the Synthedia 4 conference to offer a behind-the-scenes briefing about how GPT-3.5 and GPT-4 are used by enterprises today. Eighty percent are using generative AI grounded with their own data through retrieval augmented generation (RAG) for knowledge management chatbots. However, there are other use cases for personalized content, image generation, code generation, trend forecasting, call analytics, customer service agent assistance, and more.
Notice that Whalin also broke out the capabilities that support the use cases. Azure views content generation, summarization, code generation, and semantic search as 1.0-level capabilities enabled by generative AI. She shared several examples of 2.0-level capabilities segmented by industry. These include media workflows in telecommunications, a digital proposal assistant in manufacturing, and a smart incident manager in automotive.
The information provided here is not theoretical. Whalin referred to actual projects currently in development or production through Azure OpenAI Service today. A key takeaway from the information is that generative AI is already being applied to a broad set of use cases. You will likely recognize some of these as long-standing operational challenges that may finally get addressed because of the unique capabilities offered by generative AI and large language models (LLM) in particular.
A Shift to Owned Data and Orchestration
Whalin discussed the limitations of general-purpose bots for business use cases. Those limitations are a key reason why RAG and other approaches that infuse company data into solutions alongside LLMs aren’t just popular; they are generally necessary to achieve the objectives. Knowledge of the world is useful. Grounding in the knowledge of the business is critical.
The other trend that comes with enterprise adoption is orchestration. You must connect with other systems and services once you get beyond knowledge management and want to execute a business process. The objective may also lead to running multiple LLMs that are tuned to specific tasks. This means that orchestration, or selecting where tasks are routed and which are accepted as responses, becomes a requirement. Whalin also indicated the shift toward solutions with orchestration is a rising trend.
OpenAI vs. Azure OpenAI
Microsoft is the largest shareholder in OpenAI and also runs Azure OpenAI Service, which seemingly pits the organizations against each other when it comes to signing up customers. Whalin rejected the idea that there was an explicit conflict as OpenAI builds models and services, and Azure is there to help organizations bring those to production at scale.
There is a lot of overlap between the customers that we target for both of our services…What we have seen is if you are primarily in the earlier stage of your product development, where you're focusing on research experimentation or basically exploring the capabilities of what can you do with these models, you go to OpenAI. Now, that's not to say that Azure OpenAI doesn't provide that ability for you to experiment but we've seen that if you are in the POC stage or you're early, especially if you're coming from Academia, OpenAI is excellent.
For Azure OpenAI, what we're providing is enterprise-grade security and compliance, and we’re providing scalability. We have solid content safety models which will make sure that … if you're deploying it to millions of users, they don't see something that they're not supposed to.
If you're looking for more of a managed service approach, people usually come to us, especially given how new things are. There are considerations around throughout and latency. We understand capacity management really well. We have our own fleet of GPUs. So, all of that combined makes us really attractive to large enterprises.
A Window into Enterprise Adoption
The video above is well worth a watch. OpenAI’s rise among enterprise customers will mostly proceed through Azure. So, hearing about how businesses are using Azure OpenAI service provides a window into enterprise adoption of generative AI. OpenAI is parsimonious about sharing details regarding its business users. Azure is helping to fill in some of that information gap.
