A Timeline of Large Language Model Innovation

What's happening today builds on a rich history.

Large language models (LLM) are the workhorse behind much of the generative AI frenzy, but not everyone followed the development progress before GPT-3 in 2020 or even ChatGPT in 2022. Earlier this year, Voicebot and Synthedia put together a timeline reviewing key milestones going back to 1966, and we just updated it through August 2023.

The timeline begins at MIT in the 1960s, and the latest entries come from Meta. Most people will agree that much of the current activity owes a debt to many of the items through the 50+ year history, but special attention should be assigned to the introduction of transformer models in 2017.

Few people noticed last week that Google’s new generative AI copilot called Duet AI actually went to broad release in Google Workspace before Microsoft’s 365 Copilot. Google has been working on generative AI for a long time, and a brief review of the timeline shows the transformer research in 2017, the chat LLM LaMDA in 2021, and PaLM 1 LLM in 2022.

Similarly, NVIDIA developer Megatron, the forerunner to Nemo, with Microsoft in 2020. AI21 Labs’ Jurassic LLM from 2021, was preceded by its LLM-powered grammar and writing checker Wordtune in 2020. Meta’s entries are newer, but they are also notable.

Consider this a reference document. If you would like a higher-resolution version or PDF, you can find it here on Voicebot.ai. We intend to update it from time to time and always welcome input and feedback.

Let us know if you think we missed something important or if we added an entry that doesn’t merit the attention.