9 Jaw-Dropping Features Coming to ChatGPT - Video Demo
Greg Brockman shows how ChatGPT will change everything
Greg Brockman, president of OpenAI, just presented a Ted Talk that shows why ChatGPT is on a course to becoming an “everything app” and the virtual assistant we’ve wanted all along. During a TED Talk, Brockman demonstrated new features coming to ChatGPT and also talked about four important concepts:
How conversational interactions backed by an intelligent “assistant” make it easier for users to get what they want.
How a natural language UI will complement existing UIs and services.
How ChatGPT is trained to understand requests and what applications will fulfill the user intent.
How ChatGPT is trained to anticipate unarticulated user needs based on the conversation context.
I will break down these concepts further and provide more context around ChatGPT as an assistant in tomorrow’s post. For today, let’s focus on some new features that are going to push ChatGPT into new territory and make it even more amazingly useful than what we’ve seen so far.
Read Part 2 of this post here.
9 New Features for ChatGPT
Here are nine new features coming soon to ChatGTP that Brockman demonstrated. There were a few more that you may be able to find within his prompt demonstrations, but these seem particularly important. The video is embedded below, and we have included time stamps for each feature for easy access.
1. Images (time stamp 1:15)
According to Brockman, “We have a new DALL-E model, which generates images, and we are exposing it as an app for ChatGPT to use on your behalf. ChatGPT doesn’t just generate [text] in this instance. It also generates an image.”
Note that he says, “a new DALL-E model.” My hypothesis is that DALL-E 3 is already available in the Bing Create image generation solution. The quality appears to be higher than what you get from DALL-E 2 through OpenAI.
The update of adding image generation to ChatGPT is not particularly surprising given that the announcements and discussion around GPT-4 stressed the multimodal input and output features. Nonetheless, this will make DALL-E accessible to many more people because ChatGPT has such a large user base.
2. Memory (2:24)
You will also be able to save something for later. This appears to be a native feature of ChatGPT but it is unclear how you will access the information afterward. And it could be a plugin as well. Brockman referenced this feature but the video does not show his computer screen during that segment. Still, this is sure to be a very valuable feature.
3. Tool observability (2:33, 9:20)
You will also be able to see the plugin that ChatGPT invokes and click a button to see more information about the chain of reasoning and the sources that led to the result delivered by the plugin. By consulting the information sources, this will help you diagnose whether ChatGPT accurately understood your prompt and verify the results.
4. Automatic Tool Selection (3:05)
ChatGPT will select the right plugin for you based on the prompt. There should not be a need to select the plugin first unless you want to ensure ChatGPT employs a specific tool. This was unclear in earlier demos of plugin functionality which showed a plugin drop-down selector. It looks like you may be able to select a favorite plugin but it will be unnecessary if you trust ChatGPT to make the best choice given your request.
Automatic selection seems like a useful and necessary feature. If there are thousands of plugins, there would be no practical way to easily find what you needed. It would be a UX nightmare.
5. Integration with web apps (3:40, 4:43)
ChatGPT responses will provide URLs to web apps that will facilitate steps such as updating and confirming orders, or reviewing a Tweet before posting. The demonstration included URL links in some ChatGPT responses that could click to execute the step in another webapp such as Instacart. The Twitter integration appears like it might be native to ChatGPT and use a Zapier connection.
There has been a lot of discussion about whether plugin partners would have a path to monetiztaion. The Instacart connection from the demo shows how that might work. You could also see how subscription-based apps could provide added features to ChatGPT users as well.
6. AI-Driven Fact-Checking (8:47)
This is pretty amazing. Everyone likes to talk about hallucinations and the propensity for ChatGPT to get facts wrong, while presenting them confidently as correct.
Brockman demonstrated a “browsing” plugin that can fact-check answers generated by ChatGPT and provide an updated or verified response. The response also provides citations to double-check the veracity of the information when needed.
He invoked this feature by selecting a plugin through the drop down called “Browsing,” pasting the previous question and answer into the dialog box and then typing, “Fact check this for me.” It updated the answer with more precision and he also showed how you can review the steps the fact-checking tool executed.
It is great to have source links to enable you to fact-check. However, it is even better to have ChatGPT use AI to save time and fact-check for you. Whether you are comfortable with the “fox guarding the hen house,” is up to you. But, most users will learn pretty quickly whether this solution is reliable.
7. Analyze File Data (10:59)
You will be able to upload files, and ChatGPT will index the information and be able to answer questions about it. This was particularly impressive when the file is a CSV. However, you can see how this will quickly become broadly useful. Summarization is a great feature but being able to query structured or unstructured data from documents could be a daily use case.
8. Make Charts from a Data File (12:30)
Related to the file analysis, ChatGPT will also be able to represent data in chart form in a response. He uploaded a python script to analyze the spreadsheet and it was then able to create charts based on the data it presented.
While this was a file from his hard-drive used to analyze the data, you can see how this could become a plugin or app from an appstore you can apply when needed. The chart was not beautiful, but the next logical step would be to export data for each chart that you can use in another program that makes more beautiful charts. Or, of course, another plugin that makes beautiful charts would be even better.
9. Extrapolate a Trend from Data (13:21)
Brockman even asked ChatGPT to update a chart to project full-year 2023 data based on historical averages. The dataset for 2023 only included information through April 13th. In order to better compare 2023 with previous years, he asked ChatGPT to make a full year 2023 projection based on 2022 data through April 13th and regenerate the chart. It did this and automatically updated the title to reference the 2023 projection.
Not everyone knows how to make full-year projections from historical data. Even those that do, would require several steps to execute the task. This feature makes the task both more accessible and faster.
Your New Browser?
These features go beyond what I was expecting at this point and suggest we will once again need to update our assumptions about what is practical and possible. You can also see why ChatGPT will not remain as a demonstration app to simply show people the power of OpenAI’s large language models.
OpenAI has a unique opportunity to put a lot of functionality together in an “everything app” that may well become more important to you than your browser and finally fulfill the earlier promise of voice assistants. More on that topic tomorrow.
For now, a question. Please put your response in the comments.
What feature do you hope ChatGPT will add?
