100% retrieval accuracy on the Needle-in-a-Haystack test in a 128k token context window? I know these are test conditions, but that’s very positive news for RAG type of applications, no?
Yes, positive for knowledge assistant applications. RAG is used today in many applications because it is superior to using traditional relational DB, flat files, etc., and also because the context windows are only okay. However, my supposition is that for data you are going to use once or updates dynamically, this may eliminate the need for using RAG.
100% retrieval accuracy on the Needle-in-a-Haystack test in a 128k token context window? I know these are test conditions, but that’s very positive news for RAG type of applications, no?
Yes, positive for knowledge assistant applications. RAG is used today in many applications because it is superior to using traditional relational DB, flat files, etc., and also because the context windows are only okay. However, my supposition is that for data you are going to use once or updates dynamically, this may eliminate the need for using RAG.