Revolutionizing Chatbots: OpenChatKit v0.16 Unleashes Lightning-Fast Performance on consumer GPUs

The world of chatbots just got a whole lot more exciting with the release of OpenChatKit v0.16 by Together! With the latest update, OpenChatKit now offers users the ability to run their state-of-the-art chatbot technology on consumer GPUs, unlocking lightning-fast performance and unprecedented quality. The new 7B parameter model has been fine-tuned based on user feedback, ensuring that conversations with the chatbot are more natural and engaging than ever before.

OpenChatKit was recently launched by Together. It is an open-source toolkit for building conversational AI chatbots. It provides developers with a set of tools and pre-trained models that can be customized to create chatbots tailored to specific use cases. OpenChatKit uses state-of-the-art natural language processing (NLP) techniques to understand user inputs and generate responses. With the ability to fine-tune models based on user feedback, OpenChatKit allows developers to create chatbots that are both accurate and engaging. It is designed to be easy to use, even for those with limited machine learning expertise, making it a popular choice for businesses looking to add chatbot functionality to their websites or customer service platforms. It was presented as an open source alternative to ChatGPT from OpenAI.

OpenChatKit V0.16 Updates:

Following are some of the important updates to highlight in OpenChatKit V0.16.

1. OpenChatKit now features a brand-new 7-billion parameter model that is 8-bit quantized and optimized for consumer GPUs, enabling lightning-fast performance and efficient use of hardware resources.
2. The 20-billion parameter model has also been significantly improved by fine-tuning it based on user feedback, resulting in even higher quality conversations with chatbots built using OpenChatKit.
3. OpenChatKit now includes a new open-source dataset called "together-user-feedback" that provides high-quality instructions from end-users, helping developers train chatbots that better understand and respond to user input.
4. Thanks to contributions from the community, OpenChatKit now supports multi-GPU training and inference, allowing developers to scale their chatbot models across multiple GPUs for faster training and more efficient use of computing resources.
5. Training scripts are now available for all of the models in OpenChatKit, making it easier for developers to fine-tune the models or train their own custom models from scratch.

As per OpenChatKit, One of the most frequent requests from the community on Discord has been for a smaller model that can run more efficiently on GPUs with limited memory. The latest 0.16 release of OpenChatKit now includes a 7-billion parameter model that is quantized to 8-bit, making it possible to run the model on many consumer-grade GPUs. This new addition is sure to delight developers looking to build chatbots that can operate on less powerful hardware while still delivering excellent performance.

Announcing OpenChatKit v0.16! You can now run OpenChatKit on consumer GPUs with a new 7B parameter model fine-tuned on user feedback for improved quality. And it's fast!
Details in 🧵 ... pic.twitter.com/yOPQjeJLSo

— Together (@togethercompute) March 30, 2023

In collaboration with LAION and Ontocord.ai, Together has developed the OIG training dataset, providing a comprehensive set of examples to help developers train chatbots that can better understand and respond to user input. Since its initial release, OpenChatKit has already garnered impressive numbers, having processed over 240,000 requests through the feedback app, with more than 18,000 model downloads from Hugging Face, and over 1,200 feedback submissions from users. This is a great achievement in less than 3 weeks time of its first release. There is a lot of momentum going on here as recently there was an announcement of Introducing OpenChatKit on SambaNova Suite.

OpenChatKit v0.16 is a significant step forward for chatbot development, offering faster performance and improved quality with the new 7B parameter model and multi-GPU support. The new open-source dataset and fine-tuned models based on user feedback also ensure that chatbots built with OpenChatKit are more engaging and natural in their conversations. Try OpenChatKit v0.16 today and experience the future of conversational AI! The complete press release of OpenChatKit v0.16 is available on Together's website.