How HuggingChat💬 Compares to ChatGPT ... and How to Give it a Try

HuggingChat is the open source alternative to ChatGPT

Hugging Face launched HuggingChat yesterday as “the first open source alternative to ChatGPT,” according to company co-founder and CTO Julien Chaumond. He added that the new general-purpose generative chatbot is “powered by Open Assistant’s latest model — the best open source chat model right now — and the @huggingface Inference API.”

Open Assistant was developed by LAION, a German non-profit known for developing free large-scale AI training data sets. The specific model behind Hugging Chat is the oasst-sft-6-llama-30b, a 30 billion parament model trained on just over 150,000 messages.

Note how Chaumond set up the larger idea. OpenAI has closed API that are proprietary and tightly controlled. Many people see it as leading the generative AI market. Hugging Face focuses on open source solutions. At one time, Linux was the open source operating system kernel alternative to the proprietary (i.e., closed) software provided by Microsoft and others. Hugging Face would like to lead with open source generative AI alternatives to the closed source Microsoft … (ahem..) OpenAI.

Of course, the company founders will likely feel similarly about Google, Nvidia, Amazon, and other proprietary generative AI model providers. It might have been a fun choice to call it HuggingBard, but we all know ChatGPT is the target.

How HuggingChat Compares to ChatGPT

I did a quick evaluation of HuggingChat compared to ChatGPT’s GPT-3.5 and GPT-4 models, as well as Google Bard. Shortly after ChatGPT launched, I asked it a series of questions and compared it to Google LaMDA, which was available in a limited beta.

Below I have included one conceptual question and one fact-based question for the rapid test. First, I will show the original ChatGPT (i.e., based on GPT-3.5) response. Examples from HuggingChat, Google Bard, Perplexity AI, and the current ChatGPT with GPT-4 follow. There is a bit of reading in the first example for a few responses, but you can skim those to get the gist.

I did not include Bing Chat because it would represent a second GPT-4 text generation. Including Bing Chat in a more comprehensive analysis would be useful because it is tuned differently than ChatGPT, optimized for search, and has a real-time internet connection. However, Bing Chat is omitted here for brevity and to focus on a broader range of solutions.

Conceptual Question

How could an independent voice assistant effectively compete with Alexa, Siri, and Google Assistant for consumer adoption?

ChatGPT (December 2022)

HuggingChat (April 2023)

Google Bard (April 2023)

Perplexity AI (April 2023) - Short

Perplexity AI (April 2023) - Long Answer (this was added to show what a one-click expansion gets you from a Perplexity search)

ChatGPT (April 2023 with GPT-4)

Each of these represents the first generation of text following the prompt. Results will vary with subsequent prompts. However, they provide enough of a basis for comparison.

Google Bard misunderstood the question and refused to answer it because it inferred it was inquiring about people. This is likely a recall error as training data for Google LaMDA almost certainly includes information related to this topic.

The original ChatGPT in December gave a robust answer with three plausible strategies. The new ChatGPT with GPT-4 offered the most comprehensive response suggesting 10 different options for executing the strategy. Perplexity AI’s short response offered three plausible strategies, while the longer response included four options and some additional context.

HuggingChat offered a single, yet plausible, strategy. However, aside from Bard, it offers the narrowest response in suggesting only one strategy. In addition, the strategy HuggingChat suggested has been tried on a small scale but has never generated significant adoption. That means this might be a plausible strategy that is likely to fail.

Also, the obvious answer—which none of the chatbots picked up on with appropriate specificity—is to “invent” ChatGPT or a similar solution. ChatGPT embraces the differentiation and sharp focus on niche features that were recommended in several of the responses. It is also becoming regarded at the next-generation virtual assistant.

Factual Question

Another question genre focuses more on fact-based answers.

Why can’t I use Siri with my Amazon Echo?

ChatGPT (December 2022)

HuggingChat (April 2023)

Google Bard (April 2023)

Perplexity AI (April 2023) - Short

ChatGPT (April 2023 with GPT-4)

HuggingChat offers the briefest response to this question, but it is not quite detailed enough and could be misleading if you don’t have a lot of existing context about the devices. Perplexity’s response is good but not quite as good as the December 2022 version of ChatGPT.

The GPT-4 powered ChatGPT was even better. It was relatively succinct but also included useful context.

It may surprise you that Google Bard’s answer was probably the best. Bard’s answer included nuanced details, context, and workarounds.

HuggingChat is Fun and Adequate

Should you try HuggingGPT? Yes. It is fun and mostly accurate. Some people will also like the bias toward shorter responses. However, you can see the impact of a smaller inference model and a smaller data training dataset. ChatGPT and Perplexity AI outperform HuggingChat, and Bard does also for the factual question category.

It is unclear whether HuggingChat will be more than a demonstration project, but it is worth trying now and could become the alternative to ChatGPT.