The ChatGPT API Has Arrived and the Simultaneous Whisper API Launch is No Accident
They've also reduced ChatGPT inference cost by 90% since December
OpenAI announced today that the long-awaited ChatGPT API is now available. For some time, we have seen companies say they are using ChatGPT behind their technology stack, but the reality was GPT-3.5. Also known as the davince-003 model, GPT-3.5 was also a nice step-up from GPT-3 but lacked some of the fine-tuning of ChatGPT that many developers wanted.
Synthedia has known for a few weeks that some developers already had access to a ChatGPT API and that the availability was very limited. Frankly, I was expecting a longer beta period, but here we are.
SNAP, Shopitfy, Instacart
Snapchat’s My AI feature includes a short demo in the blog post and launched with the ChatGPT API. There are also short video examples from Shopify, Instacart, Quizlet, Shop, and Speak. You can see the heavy emphasis on shopping. What better way to pay for your new token-crunching chat solution than by selling more products or increasing the basket size?
Synthedia was recently apprised of another e-commerce solution using GPT-3.5 that outperformed fully tuned NLU models. I would expect to see a lot more experimentation in this area in 2023. Snapchat’s My AI, and Quizlet are entertainment features with constraints compared to what a user gets with ChatGPT. This seems like a logical progression of deploying domain-specific boundaries to create a more curated experience.
Beyond Chat and Bargain Pricing
The new model name is gpt-3.5-turbo, and OpenAI says it is already being adopted beyond chat applications.
It’s also our best model for many non-chat use cases—we’ve seen early testers migrate from
text-davinci-003togpt-3.5-turbowith only a small amount of adjustment needed to their prompts.
In addition, OpenAI says it reduced the cost of ChatGPT by 10x (let’s say that is 90% 😀 less expensive) that the GPT-3.5 models so many vendors switched to last fall.
The ChatGPT model family we are releasing today,
gpt-3.5-turbo, is the same model used in the ChatGPT product. It is priced at $0.002 per 1k tokens, which is 10x cheaper than our existing GPT-3.5 models.
They can reduce the price by so much because the company says it has reduced the cost of operating ChatGPT by 90% since November 30, 2022. Those “eye-watering” costs, in the words of OpenAI’s Sam Altman, were surely a key focus area as the service hit 100 million monthly active users (MAU) in less than eight weeks.
Whisper API Layers on More Value
OpenAI’s Whisper open source code has been available since the fall of 2022, and the company says it “has received immense praise from the developer community but can also be hard to run. We’ve now made the large-v2 model available through our API, which gives convenient on-demand access priced at $0.006 / minute. In addition, our highly-optimized serving stack ensures faster performance compared to other services.”
That last comment sounds like a reaction to several providers introducing hosted Whisper solutions over the past four months. Apparently, OpenAI isn’t ready to give that potential revenue stream up entirely to companies grabbing the open source code and spinning up their own models.
So, why is the Whisper API coming out now? Whisper is a multi-language speech recognition model, and a lot of developers want to provide a voice front-end interface with ChatGPT responses. Now they can grab both APIs from one provider at the same time.
