The Implication of OpenAI's False Start on Arrakis

GPT-4's output quality is superior, but it is also slow and expensive to run

The Information reported that OpenAI was working on a model upgrade to ChatGPT called Arrakis in late 2022 and early 2023. At the time, ChatGPT was using the new GPT-3.5 model tuned for the chat user experience. OpenAI already planned to upgrade ChatGPT’s performance with the GPT-4. However, a new model, Arrakis was supposed to upgrade ChatGPT again by making it less expensive to run and presumably faster at inference.

Arrakis was eventually scrapped in mid-2023 for failing to meet OpenAI’s expectations, according to the report. It may also have undermined the company’s perceived invincibility, particularly inside Microsoft.

But by the middle of 2023, OpenAI had scrapped the Arrakis launch after the model didn’t run as efficiently as the company expected, according to people with knowledge of the situation.

The Quest for LLM Efficiency

Of course, OpenAI did release GPT-3.5-turbo in March, which was positioned as a lower latency and lower priced solution. Arrakis was intended to be a new model instead of an evolution of GPT-3.5. This move offers some support to the rumors surrounding Arrakis.

The Information report positioned the Arrakis setback as simultaneous evidence of OpenAI’s competitive vulnerability and undermining Microsoft’s confidence in the team. The bigger story may be what might have been.

Time, Cost, and Quality

GPT-4 later came to ChatGPT and was immediately praised for its higher-quality outputs. However, this came at a cost. GPT-4 has high latency compared to the 3.5 model and the products' main competitors. It also costs far more than running most of the models OpenAI did not create.

The costs and latency figures have been so large that some enterprises are looking closely at alternatives. GPG-3.5 is widely used, but GPT-4 is rarely encountered outside ChatGPT Plus. The most significant takeaway from this situation is not the relationship with Microsoft. It is more likely the fact that OpenAI would have been even more dominant if Arrakis met expectations.

Cost and latency challenges make GPT-4 impractical for many business use cases. GPT-3.5 is also relatively costly and slower than the turbo model. If Arrakis could have solved these issues, there would be less incentive to try other models. Now that enterprises are considering everything from Bedrock to PaLM, Arrakis looks like a missed opportunity.

Learn More about LLMs

If you want to learn more about large language models (LLM), join me at the Synthedia 4 online conference focused on LLM Innovation this week. The event is free, but you must register. Azure OpenAI Services, Amazon Bedrock, NVIDIA, Bing Chat, Applause, and others will present.