Hey guys, welcome to Analytics Vidhya. 
Idea behind doing this video is to give you a first-hand experience of how to 
get started and perform hands-on - with the leading open-source generative models 
- using the free T4 GPU on Google Colab. In the last 1 year, the world has been 
wholeheartedly appreciative of ChatGPT and all the cool stuff we may do with it. But 
all this while, there’s been a silent revolution that has taken shape and is now ready to disrupt 
this whole domain of Generative AI. This silent revolution is nothing but the emergence of more 
than a handful of Open Source Models that have been released into the public domain. Some 
of these Open Source Models have even beaten or gotten fairly close to GPT-4 on certain 
task-specific use cases, like Code Writing. So, in this video, we are going 
to perform hands-on with three

different Open Source Generative Models:
Dolly 3B Model - released by Databricks. Falcon 7B Model - by Technology Innovation Institute (UAE). Stable Diffusion v1-5 - by RunwayML.
To ensure that all of us are able to follow along, we’ll be running these models on the free version of Google Colab - using their free T4 GPU. In fact, that’s the precise reason why we have chosen these smaller-sized models, so that we may do our inference seamlessly. Yes, smaller model means deteriorated performance. But the idea here is to know the possibilities rather than pursuing perfection. Makes sense, right? Just one last thing, guys, do us a favor by subscribing to our channel. It’s absolutely free for you, but it motivates us big time. So please, please do that. Now let’s jump into the hands-on.
[Dolly 3B Model - by Databricks]

So, the first model we are going to perform hands-on with is the Dolly Model. It’s an Instruction-tuned model that we are using - having 3 billion parameter. For pulling the model from Huggingface, we are going to use their Transformers library. This library provides APIs and tools to download state-of-the-art pre-trained models for performing hands-on within our system environment, which is Google Colab in this case. Additionally, we are using sentencepieces commonly used for text tokenization use cases in Natural Language Processing. And accelerate, which allows for faster GPU computing in Machine Learning. Don’t worry too much about these two. Then we are importing these bunch of modules: PyTorch which allows Tensor computation with strong GPU acceleration. transformers pipeline which is a fast way to use pre-trained models for inference and Os which is a standard Python Module Thereafter, we load the model using the

Pipeline function.
For this, we create an instance of a Dolly Pipeline.
This is the model that we are specifying, which is: databricks/dolly-v2-3b.
This next line sets the data type used by PyTorch to process data.
torch.bfloat16 represents the 16-bit binary floating-point data type.
Using a lower precision data type like bfloat16 can speed up neural network computations and reduce memory usage. trust_remote_code=True: This setting allows the pipeline to trust remote code that will be downloaded into our runtime environment.
device_map=auto: This line specifies the device map for processing. auto typically means that the pipeline will automatically select the best available hardware (CPU or GPU) for processing, based on system's capabilities. Given we have a GPU available in Colab, that will be mapped and utilized. Alright, now let’s run this code. It might take some 2-3 minutes to execute. So just wait for it.

Moving on, we have this helper function for prompting the dolly model. Within the function, we are specifying this prompt. Then, we are calling the dolly_pipleline that we created above to generate response on our prompt. And finally the helper function returnt he Model response. Let’s initialize this function now.

Alright, now comes the exciting part, where we are going to prompt our Dolly Model. Let’s say, we ask this: Isn’t this cool? Now, I’m sure you would start thinking about what you could do with this power that you possess now. All for free. For that, I have a small glimpse to share with you, as to what all is possible. Now, on top of this Dolly Pipeline, let’s build a Generative AI Application, called Your AI Physics Teacher. For that, first up we install gradio. And then, we write this gradio app code.

Now let’s query our App.

So this was all I had on Dolly Model. Now similar to this, you may also perform hands-on with the Falcon 7B model. In fact, I’ve kept this ready-to-use code for you to do this. Just make sure you restart your run-time before you get started with this Falcon hands-on. There’s one Model still left, which is the Stable Diffusion v1-5 - that I’ll be covering in the second part of this video. So do look out for that next week. If you haven’t subscribed to our channel yet, please do it so you get notified whenever that Stable Diffusion hands-on video is live. Happy learning to you. Bye!!

Run 3 Open-Source LLMs on Google Colab - for FREE ⚡️ Top Generative AI Model Hands-on (Hugging Face)

For those of you, who are wondering what is “Analytics Vidhya”, “Analytics” can be defined as the science of extracting insights from raw data. 

The spectrum of analytics starts from capturing data and evolves into using insights/trends from this data to make informed decisions. 

“Vidhya” on the other hand is a Sanskrit noun meaning “Knowledge” or “Clarity on a subject”. Knowledge, which has been gained through reading literature or through self-practice / experimentation.

Analytics Vidhya provides a community-based knowledge portal for Analytics and Data Science professionals. The aim of the platform is to become a complete portal serving all knowledge and career needs of Data Science Professionals. 

7805

Run 3 Open-Source LLMs on Google Colab - for FREE ⚡️ Top Generative AI Model Hands-on (Hugging Face)

Run 3 Open-Source LLMs on Google Colab - for FREE ⚡️ Top Generative AI Model Hands-on (Hugging Face)

Scripts