OpenAI Readies Open-Source Model as Competition Intensifies

OpenAI is reportedly gearing up to unveil a new open-source language model, a development informed by the increasing competition in the field of open-source language models. First reported in The Information, sources close to the matter suggest that this move is a response to the burgeoning development of rival open-source language models, some of which may threaten to undermine OpenAI's ambitious plans. While OpenAI is unlikely to release a model that is competitive with GPT-4, its decision to fast-track an open-source version may speak to an attempt to control the emerging narrative. 

The Rapid Evolution of Open-Source Language Models

Since Meta’s LLaMA open-source language model first leaked in February, the open-source community has driven a renaissance of innovation on top of the foundational language model. Google AI engineer Luke Sernau’s leaked internal memo, which claimed “we have no moat, and neither does OpenAI,” has driven vigorous debates within the AI community about the future of chatbots and who holds the true advantage. 

New techniques have enabled researchers to fine-tune and improve Meta’s language model for just hundreds of dollars, and free AI models are now “reasonably close” to Google’s Bard and OpenAI’s ChatGPT, says Ion Stoica, a professor of computer science at University of California, Berkeley. Using just $300 in computing power, Stoica’s team trained Vicuna, an improved version of LLaMA, on 70,000 public ChatGPT conversations and claims that their model delivers 90% of the quality of ChatGPT and Bard.

Chatbot AI's Existential Question

Sernau’s leaked memo raises a vital question that resonates with all AI companies: "Who would pay for a Google product with usage restrictions if there is a free, high-quality alternative without them?” He argues that Meta's open-source approach generates a reservoir of innovation that it can tap into to refine its business AI models. In contrast, Google and OpenAI maintain a guarded stance on their AI systems, with Google even discouraging its research team from publishing additional AI papers.

This hasn't deterred other players from joining the fray. Firms like Anthropic and Inflection AI are building their private AI models, while Stability AI, the creators of Stable Diffusion, introduced an open-source language model competitor in April. This proliferation of models, both proprietary and open-source, motivated Stoica to create the Chatbot Arena, a platform that measures the performance of various models using the Elo rating system.

How the Future Could Play Out

Predictions about the future of language models are diverse and intriguing. Stoica suggests that Google could leverage its vast user data trove to maintain a significant edge over open-source models. Contrarily, Sernau's memo proposes that open-source models could potentially surpass proprietary models as they "are used and created by people who are deeply immersed in their particular subgenre, lending a depth of knowledge and empathy we cannot hope to match.”

Companies may opt to fine-tune open-source models to their specific needs rather than employing a generalized language model like OpenAI's. This trend is evidenced by Bloomberg, which announced in March that it used its data to develop BloombergGPT, a language model tailored to finance.

If OpenAI does release their open-source model, it would not be the company’s first such release; GPT-1 and GPT-2 are both open-source. Even Google’s earlier models, including its T5 translation model and BERT, are open-source as well.

Details about the internal discussions at OpenAI remain murky. However, one thing is evident: OpenAI is determined to avoid a repeat of its DALL-E 2 episode. While initially hailed as a groundbreaking image generation AI, DALL-E 2 has since been overshadowed by open-source alternatives like Stable Diffusion. As reported by The Information, even OpenAI’s employees were taken by surprise by this turn of events, and they're keen on preventing a similar setback with ChatGPT.