12 Things that Matter from OpenAI Dev Day - Including 2M Developers and Personal Bots
12 Things that Matter from OpenAI Dev Day - Including 2M Developers and Personal Bots
Personal ChatGPT, Faster and Cheaper GPT-4, and a New Bot World Order
OpenAI concluded its first Dev Day keynote a short time ago, and we followed that up with a special edition of the GAIN Rundown. Our guest experts had a lot of interesting insights to share and offered an important perspective on how the new features and products will impact the generative AI market. I recommend you watch the video discussion here. If you want to listen to just the winners and losers section, go here.
ChatGPT has 100 million weekly active users (WAU). OpenAI has two million developers using the APIs. Over 92% of the Fortune 500 are using OpenAI technology. GPT-4 Turbo has been updated on training data through April 2023 and is now used for ChatGPT.
Inserted between the many announcements at OpenAI’s first Dev Day event keynote were several interesting statistics that shed more light on the company’s market momentum. But the star of the show was a compelling new product and numerous features for developers.
The entire Dev Day video is also on YouTube, and you might find that it jumps around a bit. There are two main segments:
New features and announcements for developers that use OpenAI’s APIs.
The new GPTs product offers the ability to build a customized ChatGPT bot for personal or business use and there will be an app store of sorts.
There is a lot to digest as OpenAI is both announcing new features and indirectly clarifying its strategy based on the product focus. As to the vibe onsite, two onsite attendees sent me their hot-takes:
Energy here is enthusiastic and uncritical.
Most exciting...Custom LLMs. A good first step. Second step - not mentioned - standardization for labeling and auditing LLMs.
Focusing LLMs for brands seems to be what business wants.
Assistants API is a great evolution of plugins for devs
Multimodal apps will be much more common now that they are in same library
Revenue sharing is important
Here are 12 key announcements we saw and one that we didn’t during today’s keynote.
1. No-code GPTs
GPTs are personalized versions of ChatGPT for individual users. The early leaks referred to these as MyGPT, Gizmo, or Agents. This segment of the keynote was the most in-depth.
Users create a custom GPT bot using a no-code form building simply by prompting a natural language description. They will also be able to upload information that can be used as source material and grounding for answering questions. That can be combined with other OpenAI services, such as internet-connected web browsing for search and image generation leveraging DALL-E. These GPTs become more customized over time and serve as personalized digital assistants.
Anyone can easily build their own GPT—no coding is required. You can make them for yourself, just for your company’s internal use, or for everyone. Creating one is as easy as starting a conversation, giving it instructions and extra knowledge, and picking what it can do, like searching the web, making images or analyzing data.
These are not fully autonomous agents but can be programmed with some agentive scope of authority to fulfill tasks on a user’s behalf.
2. GPTs Store and Monetization
Starting today, you can create GPTs and share them publicly. Later this month, we’re launching the GPT Store, featuring creations by verified builders. Once in the store, GPTs become searchable and may climb the leaderboards. We will also spotlight the most useful and delightful GPTs we come across in categories like productivity, education, and “just for fun”. In the coming months, you’ll also be able to earn money based on how many people are using your GPT.
The ChatGPT app store is coming. Creators will be able to try their hand at applications backed by large language models (LLM) and benefit from easy access to the 100 million WAUs. OpenAI did not offer details on how monetization will be shared with creators of popular apps, how much revenue sharing will take place, or the time frame for implementation. However, this development will surely intrigue a lot of users.
3. GPTs Connections (services integrations)
If you have software development skills, you will have the opportunity to add even more features to your GPT.
In addition to using our built-in capabilities, you can also define custom actions by making one or more APIs available to the GPT. Like plugins, actions allow GPTs to integrate external data or interact with the real-world. Connect GPTs to databases, plug them into emails, or make them your shopping assistant. For example, you could integrate a travel listings database, connect a user’s email inbox, or facilitate e-commerce orders.
4. GPT-4 Turbo Has a 128K Context Window
The personal GPTs are sure to garner most of the attention. However, the new GPT-4 Turbo model may be the most significant announcement. OpenAI CEO Sam Altman said it was faster, but the wow factor moment was the 128K token context window.
A lot of people have takes that require much longer context length. GPT-4 supported up to 8K and in some cases up to 32K context length. But we know that isn’t enough for many of you and what you want to do. GPT-4 Turbo supports up to 128,000 tokens of context. That’s 300 pages of a standard book, 16 times longer than our 8K context. And in addition to the longer context length, you’ll notice that the model is much more accurate over a long context.
5. GPT-4 Turbo Offers More Control
Altman also expanded on new control features. A key concern related to LLMs is their unpredictability. OpenAI is addressing this with several new features.
We’ve heard loud and clear that developers need more control over the mode’s responses and outputs. So, we’ve addressed that in a number of ways. We have a new feature called JSON load which ensures the model will respond with valid JSON. This has been a huge developer request. It will make calling APIs much easier….You can now call many functions at once and it will do better at following instructions in general.
We are also introducing a new feaure called reprouceable outputs. You can pass the seed parameter and it will make the model return consistent outputs. This, of course, gives you a higher degree of control over model behavior.
The reproducible outputs might be the most intriguing of these features. Many companies achieve this objective today by intercepting the prompt and assigning the response job to natural language understanding (NLU) AI models, bypassing the LLM. Passing the seed information will enable the LLM to perform this job, according to Altman.
If this feature produces more reliable and consistent results, it may calm the fears of company managers, lawyers, and compliance professionals. Well, that may depend on whether this reduces response variability or eliminates it. It will be a feature worth monitoring. It is available today in beta access, according to the announcement.
6. GPT-4 Turbo is Cheaper
OpenAI is also reducing the price of GPT-4 with the Turbo release. Input token costs for GPT Turbo are declining by two-thirds compared with GPT-4 and will be $0.01 per thousand tokens. Output tokens will be $0.03 per thousand. That is for the 128K context window compared with 8K for the legacy solution. The declines are even steeper for the limited-access 32k model. GPT-3.5 Turbo prices are also reduced.
7. Internal-Only GPTs
Companies will also be able to implement GPTs that are only accessible to their employees. This will be for both no-code and developer-led customizations. It is the key announcement related to ChatGPT Enterprise and appears to be available only to those users. That makes sense, as the administration features require the enterprise-class application.
Since we launched ChatGPT Enterprise a few months ago, early customers have expressed the desire for even more customization that aligns with their business. GPTs answer this call by allowing you to create versions of ChatGPT for specific use cases, departments, or proprietary datasets. Early customers like Amgen, Bain, and Square are already leveraging internal GPTs to do things like craft marketing materials embodying their brand, aid support staff with answering customer questions, or help new software engineers with onboarding.
Enterprises can get started with GPTs on Wednesday. You can now empower users inside your company to design internal-only GPTs without code and securely publish them to your workspace. The admin console lets you choose how GPTs are shared and whether external GPTs may be used inside your business.
8. Custom Models
Not only will enterprises be able to create custom user experiences through GPTs, they will also be able to create custom models. GPT-4 fine-tuning will be available and offer some customized performance for the first time on OpenAI’s flagship foundation model. In addition, for customers “with extremely large proprietary datasets—billions of tokens at minimum we’re also launching a Custom Models program, giving selected organizations an opportunity to work with a dedicated group of OpenAI researchers to train custom GPT-4 to their specific domain.”
So, OpenAI will offer professional services in addition to APIs for these customers. That is a development I had not anticipated.
9. GPT-4 Turbo Gets Senses
OpenAI’s APIs are graduating beyond modality segmentation. GPT-4 Turbo offers vision (image input features), DALL-E image generation, and text-to-speech (TTS) for spoken audio output. We have already seen some of the image generation capabilities in ChatGPT, and there is a beta version of Bing Chat that offers image uploads.
The TTS voice quality is pretty spectacular, by the way. And the Whisper speech recognition also received an upgrade.
10. GPT-4 Copyright Shield
OpenAI is also following the lead of Adobe and Google by providing copyright protection for its users. However, you might note the limitations. Google also has several limitations related to the status of the product and how the customer accesses the generative AI technologies. OpenAI is only offering protection for ChatGPT Enterprise direct-access API customers.
We will now step in and defend our customers, and pay the costs incurred, if you face legal claims around copyright infringement. This applies to generally available features of ChatGPT Enterprise and our developer platform.
11. Assistants API
The rumors I heard from developers in advance of the event about agents were related to the Assistants API. It is a response to the popularity of the open-source Auto-GPT project. The documentation is more helpful than the blog post for understanding these features.
The Assistants API allows you to build AI assistants within your own applications. An Assistant has instructions and can leverage models, tools, and knowledge to respond to user queries. The Assistants API currently supports three types of tools: Code Interpreter, Retrieval, and Function calling. In the future, we plan to release more OpenAI-built tools, and allow you to provide your own tools on our platform.
At a high level, a typical integration of the Assistants API has the following flow:
Create an Assistant in the API by defining it custom instructions and picking a model. If helpful, enable tools like Code Interpreter, Retrieval, and Function calling.
Create a Thread when a user starts a conversation.
Add Messages to the Thread as the user ask questions.
Run the Assistant on the Thread to trigger responses. This automatically calls the relevant tools.
This will enable developers to more easily add complex, multi-step capabilities to their applications. The Assistants API can enable agent-like features. It is a building block for developing digital agents that can complete tasks without requiring imperative-style user instructions. However, it is not precisely an agent as you would describe a digital assistant with agency.
The agency granted is limited to accessing a few services based on a request. This is not about proactively serving the user. We will have to wait for that. In the meantime, the copilot-oriented features referred to as agents will be useful.
One final note on agents. This is not the path to artificial general intelligence (AGI), even though there are already headlines to that effect.
12. No Talk of Plugins
ChatGPT Plugins were all the rage when they debuted in April 2023. They might as well have been Voldemort today. This is not surprising for two reasons. First, the roll-out has not gone as well as expected because integration with hundreds of third-party applications, and introducing a new UI, and figuring out arbitration and orchestration of multiple applications is not easy.
This roadblock was predicted at the time by my podcasts with Adam Cheyer and Dag Kittlaus, two of the co-founders of Siri (acquired by Apple) and Viv Labs (acquired by Samsung and became New Bixby). These digital assistant pioneers had implemented the third-party plugin model twice and shared that there were technical issues, but the bigger obstacles were business processes and competing objectives.
Synthedia has learned that the number of Plugins was frozen at around 800 as OpenAI rethinks its strategy. Plus, many of those Plugins can probably now become GPTs. It is unclear whether Plugins will be abandoned or re-imagined. The architecture works for applications and is supported by Microsoft, though it has modified the approach. For now, Plugins are on the back burner.
The key mention of Plugins was in one of the blog posts where OpenAI suggested the Plugin makers should convert them to GPTs.
The design of actions builds upon insights from our plugins beta, granting developers greater control over the model and how their APIs are called. Migrating from the plugins beta is easy with the ability to use your existing plugin manifest to define actions for your GPT.
And, of course, one developer pointed out that the Assistant APIs are better for the Plugin developers.
OpenAI’s Platform Expansion
OpenAI is a platform company. The APIs are designed to offer services to application developers, and it will surprise no one that the company added new features to make the platform more attractive in terms of capabilities and cost. However, a year ago, few people would have guessed OpenAI would aggressively expand an application platform in addition to its application services.
The world has changed, and OpenAI has changed. The product that made OpenAI central to the generative AI revolution and its biggest driver of revenue is ChatGPT. So, logically, the company is working on adding more value for users.
Despite the terrible GPTs naming, OpenAI is tapping into proven user demand. Character.AI has built a large, loyal user base for its custom LLM-backed assistants (or are they companions). Thousands of companies have built custom assistants based on NLU models, and thousands more are experimenting with LLM-backed solutions. OpenAI is making it easier and bringing a no-code flavor to customization.
OpenAI will not control the entire generative AI or LLM market. However, it is seizing its moment and expanding into areas that were openings for other foundation model and application developers. It intends to control its own destiny, to a degree, and create a much larger market size for its technology. It also looks like the business and productivity segment could continue to be dominated by OpenAI. There is less and less room for other companies to establish differentiation. OpenAI’s large user base and developer ecosystem is about expand again.
It is important not to put too much stock into developer events and product launches. The ideas and packaging always make the news seem like it will have a significant immediate impact. In this case, the announcements were understated, but I suspect OpenAI just did itself a great service and solidified its market lead.
Leslie Pound told me that while sitting in the room with the other developers, she felt like an old GPS system, and every 15 minutes was “re-calibrating.” While the presentations and breakout sessions were presented merely as exciting new technology, it seemed a lot more like a “sea change.”
