OpenAI Acquires Rockset and Highlights Where Generative AI is Headed Next
Welcome to the rise of serverless RAG
OpenAI announced Friday it was acquiring Rockset, “a leading real-time analytics database that provides world-class data indexing and querying capabilities.” Rockset was founded in 2016 by two former software engineers at Facebook. The company’s focus was “cloud-native” databases, and it moved into serverless analytics and microservices three years later. In April 2023, the company introduced serverless vector search with a demonstration that leveraged an OpenAI embeddings model.
Rockset has raised $117 million in venture funding to date, according to Crunchbase, with Sequoia and Greylock investing in multiple rounds. The company raised $40 million in 2020, and $51 million in Q3 2023.
The acquisition announcement was accompanied by FAQs that indicated Rockset will be shutting down access to its software and working to transition customers to alternative providers. Month-to-month contract holders have until the end of September to transition, while free trial users will be unable to upgrade or use the service going forward. The entire Rockset team will be integrated into OpenAI, which will have exclusive access to the company’s technology.
Serverless RAG
You are probably familiar with retrieval augmented generation (RAG) if you are following the evolution of the enterprise generative AI market. It has quickly become the most popular technique for grounding AI model responses to corporate data. The impact is a lower “hallucination” rate, more precise answers, and explainability via citations. Serverless RAG emerged over the past year to address scaling challenges associated with vector databases.
Pinecone introduced a serverless RAG product in January 2024, along with claims of 50x cost savings. Existing vector store services typically charge users for peak compute capabilities whether or not they are using it. Serverless RAG only charges users for computing resources when queries are active and uses less memory for indexes. Turbopuffer is another software package that offers serverless RAG indexing, though there appear to be some differences between the three models.
Phases of Data in Generative AI’s Evolution
Over the past year, the application of data to foundation model performance improvement has experienced several phases. We first saw the scaling phase, where more data tokens used in training was the dominant strategy. That was followed by the data curation and data enhancement phases that focused on data quality. The Rockset acquisition is related to a new phase related to data access and data ops.
This phase is less about making the models better and more about improving the user output when AI foundation models are connected to enterprise data. Rockset is a step in that direction and also represents the next wave of RAG innovation and adoption.
Cohere was once positioned as an OpenAI alternative and general-purpose AI foundation model developer. It shifted its focus to data retrieval in mid-2023 and now promotes the tight integration of its large language models (LLM) with RAG. This shift reflects the first wave of enterprise adoption. Everyone want to use LLMs with their own data. RAG is the simplest and most reliable way to meet this objective today. While Cohere is not using serverless RAG today, its Command R+ models highlight the enterprise knowledge retrieval focus.
OpenAI has focused so far on delivering strong AI foundation models and has left the RAG work up to developers that integrated knowledge retrieval solutions. With that said, OpenAI already has some RAG/vector features in its GPTs and through the Assistant API. Rockset will provide additional in-house expertise and tooling to enable more capabilities directly and at a lower operating cost.
Many thanks to our title sponsor.
