Introducing Dolly: Databricks' Affordable LLM with Impressive Instruction Following Capabilities Similar to ChatGPT

Ever since OpenAI's ChatGPT is released, the tsunami of Artificial Intelligence (specially Generative AI and LLM) is getting bigger and bigger with each day passing. Every technology company is trying to grab a pie of it. In last couple of weeks, we have seen so much innovation in this space and new opensource releases like OpenChatKit from Together, Alpaca from Stanford University. Now, it's Databricks that is coming up with their LLM. 

Databricks, a leading provider of advanced analytics and AI solutions, has recently introduced a groundbreaking innovation in the world of language models - Dolly. Built on the latest AI technology, Dolly is a low-cost language model that exhibits an exceptional ability to understand and follow instructions, similar to the widely acclaimed AI model ChatGPT. With Dolly, Databricks aims to democratize access to advanced language modeling capabilities and empower businesses and individuals to harness the full potential of natural language processing.

As per Databricks research, transforming an outdated off-the-shelf open source large language model (LLM) into a powerful instruction-following machine similar to ChatGPT is not only possible but also surprisingly easy. By training the model with high-quality data for just 30 minutes on a single machine, they were able to imbue it with remarkable capabilities that were once the exclusive domain of much larger models like GPT-3, which boasts 175 billion parameters. 

Interestingly, Databricks model, which they named Dolly, only has 6 billion parameters, yet it can still perform complex instruction-following tasks. Databricks have made the code for Dolly freely available as an open-source resource and have demonstrated how it can be recreated on Databricks. Databricks believe that models like Dolly have the potential to democratize LLMs by making them more accessible and affordable to businesses of all sizes, thus enabling them to customize and improve their products with ease.

Dolly is a new and very cost-effective LLM that possesses impressive instruction-following capabilities similar to ChatGPT. Dolly is created by slightly tweaking an existing 6 billion parameter model from EleutherAI and introducing instruction-following capabilities like brainstorming and text generation, which were absent in the original model, through the use of Alpaca data.

After developing Dolly for two years, Databricks also conducted a thorough evaluation of its instruction-following abilities based on the same criteria described in the InstructGPT paper, which ChatGPT is based on. The evaluation revealed that Dolly exhibits many of the same qualitative capabilities as ChatGPT, including text generation, brainstorming, and open Q&A. These results demonstrate the remarkable potential of Dolly, low-cost LLM in the field of natural language processing and its ability to perform complex language tasks with impressive accuracy and efficiency. 

To facilitate the use of Dolly, Databricks have made a simple Databricks notebook freely available as an open-source resource. With this notebook, you can easily build Dolly yourself on Databricks, allowing you to leverage its powerful instruction-following capabilities. This Databricks notebook is available on github.

Dolly presents an exciting new prospect for companies seeking to develop their own instruction-following models at a lower cost. With its ability to exhibit ChatGPT-like instruction-following capabilities, Dolly has the potential to transform the field of natural language processing by making advanced language modeling accessible and affordable to businesses of all sizes. Companies can now leverage Dolly's low-cost framework to create customized language models that meet their specific needs, empowering them to innovate and improve their products with ease. More information about Dolly is available on Databricks blog.