Explore the cutting-edge intersection of language and technology as Rishabh Kumar, Twilio aficionado and Microsoft MVP, joins us to shed light on the transformative capabilities of Langchain interwoven with Azure AI Search. Prepare to be astonished by the boundless possibilities of applications fueled by large language models, as Rishabh unveils how they're reshaping the landscape far beyond conventional chatbots. We navigate through the innovative use of Azure AI Search within Retrieval-Augmented Generation pipelines, spotlighting a revolutionary YouTube assistant bot that not only condenses but also summarises lengthy video transcripts.
Explore the cutting-edge intersection of language and technology as Rishabh Kumar, Twilio aficionado and Microsoft MVP, joins us to shed light on the transformative capabilities of Langchain interwoven with Azure AI Search. Prepare to be astonished by the boundless possibilities of applications fueled by large language models, as Rishabh unveils how they're reshaping the landscape far beyond conventional chatbots. We navigate through the innovative use of Azure AI Search within Retrieval-Augmented Generation pipelines, spotlighting a revolutionary YouTube assistant bot that not only condenses but also summarises lengthy video transcripts.
Hello, welcome to Microsoft Community Insight Podcast, where we share insights and stories from community experts to stay updated in Azure. My name is Nicholas and I will be your host today In this podcast. We will dive into blockchain with Azure AI Search. But before we get started, I remind you to subscribe to our podcast so you never miss an episode and it will help us reach more amazing people like yourself. So today in this podcast episode, we will have the amazing Rishabh Kumar. Can you start by introducing yourself, please?
Speaker 2:Hi there. Thank you, nicholas, for having me. I'm Rishabh. I work at Twilio as a staff developer evangelist and I'm also a Microsoft MVP and, yeah, I love building cloud DevOps and, recently, ai stuff. So you can find me online. Most of you have probably seen me around the Twitter or the LinkedIn space and even youtube, but you can find me at reshub and cloud and, yeah, it's a pleasure to be here, right yeah, so today our theme of this episode is langchain, so we're gonna just talk about what it is.
Speaker 1:So first question is what is LanChain and how can we use it with AI search Rishabh?
Speaker 2:Sure. So LanChain is an open source framework that lets you build LLM powered applications. Don't just think you know simple apps where you're integrating OpenAI's API and now you can chat with a large language model, just like ChatGPT, but within your apps. But the extensibility of what you can build is just, you know, there's no limit because of the tools, the agents, the APIs that are available through langchain, and we can also, you know, go over the offerings that they have within their tools if we do want to do that. But yeah, you basically have the ability to build chatbots and agents and other things that are powered by AI using different large language models and things like Azure AI Search, which I recently have been exploring, and I think your question was how it works with Azure AI Search. There's amazing documentation available, both from Azure on Azure AI Search, its features and how you can do different things Azure on Azure AI Search, its features and how you can do different things. But also LanqChain has an Azure AI Search doc that's available, which I used to integrate it into my project.
Speaker 1:That's brilliant. So let's dive into some of the functionality of LanqChain. So what are some of the use cases of where LanqChain and AI Search have been successfully applied?
Speaker 2:Sure. So in my personal experience, what I have used Azure AI Search for is mostly RAG, which stands for Retrieval, augmented Generation, kind of pipelines, right, and that's where I think it's the best offering because of the generous free tier that you get with azure ai search. And what I have built with that is you know, I've built a youtube assistant bot where you know it gets the transcript of the video. So you basically paste a youtube video url and you ask questions to that YouTube video and then it does a rag so you get the relevant context from the video based on your question and then it's able to answer your question based on just that video. So think of you know podcasts that are three hours or four hours long, but you don't have the time to just go through the entire video. You can ask, hey, what did Person X say on this podcast about AI, let's say and then you'll get all the bullet points from this chatbot, which is amazing because it saves me time. It summarizes videos and podcasts that I'm interested in but I don't have the time to watch them, and that is like one example.
Speaker 2:But Azure AI search what I found is you also have you know capabilities of within RAG, you have relevant or relevancy score. So just to give you a context, what RAG is basically? So large language models and you might have seen this with you know chat, gpt or Gemini or whatever you use is when you ask them questions that is too recent, you'll get a response like hey, I don't know, because this hasn't happened yet, because they were trained until 2023 or 2022 or 2021. With GPT-3, it was data until 2021. So it doesn't know about current affairs or things that are happening right now.
Speaker 2:But what you can do is you can provide that context to the large language model so you can say hey, just like with my YouTube assistant, example you can say hey, here's the whole script of the YouTube video, which you might not know of because it just was released three days ago, but I want you to answer this specific question. So you can imagine that a YouTube script for a three-hour long video is going to be huge, right? Yeah, so we can't pass all of that to OpenAI or to Azure OpenAI services because they have token limits, right? You have a certain amount of context that you can provide and you need to be within those token limits, and that's where chunking and all those concepts come in. Is you break down that three-hour-long transcript into smaller chunks of documents and you store that in a vector database which Azure AI Search functions as? So the SDK for Azure AI Search, whether you're using directly or you're using it through tools like LanChain or Semantic Kernel is you have that ability to break down a huge document into smaller chunks, store it within Azure AI Search and you have the ability to do search on these documents. So, let's say, we are talking about this podcast, right?
Speaker 2:Once this podcast is released, it will have a transcript which is I don't know, let's say, 50 pages long, which is I don't know, let's say 50 pages long, and we'll break it down to single page, which will be 50 individual pages. That will sit inside and it can be any storage. It can be Azure blob storage, but Azure AI Search has capabilities to handle that for you. And then you can basically, when I ask, hey, where does ReSharp work? Right. So what would happen behind the scenes is it'll go and look at documents wherever ReSharp's workspace was mentioned, which was basically in the introduction of this podcast and it'll grab that specific page and use that as context and pass it along your question to the large language model. And now the large language model knows that we shall work set to you based on this page and I'll get back that response to the user.
Speaker 1:So that's the same as system prompt, then in Azure OpenAI.
Speaker 2:Sorry, did you say system prompt?
Speaker 1:Yeah, is that the same how LanChain uses context?
Speaker 2:That is a good question. I think when you're building a chatbot, you can have it within the system prompt, and if you're building something, you know, like a question answer machine or where you don't need the, you don't need the chat capabilities, but it's more of a question and answer situation. Yeah, you, you have that within the prompt template. So within the prompt template you'll say hey, here are the Doc's that are relevant to this question. But yeah, okay, that's brilliant.
Speaker 1:You had the right idea In your experience. How does long chain and AS search impact? Like different, like semantic search functionality? Is it the same? That's a good question.
Speaker 2:If you have your semantic search and then long chain, like I'm not sure they have a functionality called semantic search or higher let me see I'm gonna dive into my project, but I do feel like there are some functionalities you know, like similarity search, hybrid search, that you're able to utilize within your application. I don't know about semantic search.
Speaker 1:No, no worries, so let's ask a different question. So, in your experience, what challenges do we face when using AI Search with Langchain? What are some of the challenges you came across?
Speaker 2:That's a good question. I would say my experience so far has been really good, even though it has been only for a short amount of time. It's only been like a month that I started using Azure AI Search. I've been using Langchain for quite a few months now, documentation-wise, I think, with examples. I think Langchain and Azure AI Search can. I might even contribute with some examples. I think we can have more robust examples of different concepts and cover them, whether that's on the LanqChain docs or whether that's on the Azure AI Search docs. But I was able to get it configured pretty quickly, which is nice, and I was able to call different methods, which is nice, which means you know both of those teams have done great jobs when it comes to covering the documentation side. But I think with examples it would make it, you know, even better, like, hey, if you want to do this, this, you know, here's an example app that we built and I I'm pretty sure that it's already in pipeline and they're coming up with that okay, so no challenges launching.
Speaker 1:So launching, let you like integrate different services, like slap bot, like slap zero and things and just and github as well. You just connect it without, without writing more complex iphone code on top of it exactly, exactly, so you can have your rag running through azure ai search.
Speaker 2:But you can, on top of that, if you want, you know, bing, search um capabilities or wikipedia to do do some more thorough research on your answers, or even something like Twilio. If you want to send a text message or voice call, you have those tools built in with the land chain.
Speaker 1:How does in your experience when working with land chain? How does it work with different contexts like text and images to audio in a way like normal, I just search. Is it the same method?
Speaker 2:I haven't worked with uh media yet, so my my exposure has been just text. Um, I worked with like different different types of text, so you know PDFs, but I think the extraction is still the same. It's just that it's text in form of PDFs, it's text on a website, so you extract that and load it into your pipeline. But yeah, I haven't worked with like video or images that, that's, that's interesting yeah yeah, it's an interesting question I thought of.
Speaker 1:So it's just pretty much like pdf. You just put it like in github and stuff and just exported it yeah, yeah, yeah.
Speaker 2:So you can put it in github, um or wherever it's accessible. So if it's in your blobs or like azure storage account, um, you'll need those connection strings right to bring in those pdfs, um, or and then chunk it down to smaller pages and then upload it.
Speaker 1:Yeah okay, so, so thanks. So, in your experience, what are some of the projects that you use Langchain for and that's been successful, or is it just POC at the moment?
Speaker 2:Yeah, so there's this talk that I gave about introduction to Langchain as the framework. Yeah, so I have two basic examples within that talk to cover just the concepts and how LanChain works. One of them is very basic. It's just like a pet's name generator, where you select what kind of pet you have and provide a color of your pet and it gives you, like cool names, which is very basic. But then the second one is youtube assistant, um, which is like if you go to free code camp youtube channel and search for langchen course, um, I've included that project within within that course and you can kind of see the capabilities you have of building something like that. That might help you.
Speaker 2:The other thing that I built and it uses Twilio is a summarizer bot. So let's say you have an article, right, and I'm notoriously guilty of bookmarking tweets and articles that I want to read later, but you know that most of the times you don't go back and read them. I try my best to go back and clear up my bookmarks and stuff and go through them. So what I built is you basically paste a link to that article and it sends you a summary of that article as text message. So because now I have a notification on my phone saying that, hey, you want to read this article, but if you didn't, here's the summary. So it has like bullet points, which which is again built with you know langchain azure ai search and tulio um sms okay, so I take it, is that built, is that stored on your github, then?
Speaker 1:which have? So for others it is private for now.
Speaker 2:Um, but yeah, the the the goal is to have that available publicly. And yeah, you can check out my GitHub. I have other Lantern examples. Just type Rishabh Kumar 7 GitHub and you'll get my GitHub. And then you can type in Lantern in my repositories and you'll see all of the langchain repositories here.
Speaker 1:That's brilliant. So, as this episode is coming to an end, how can someone get started with langchain and integrate it with Azure Search?
Speaker 2:Good question. So if you're a Python dev which I use, so langchain an important point is is only available in Python and JavaScript slash I use. So Langchain an important point is is only available in Python and JavaScript slash, TypeScript, right. So if you're a Python dev, you can go to Langchain or you can search it. Search in your browser, langchain Python, and it will take you to the homepage of the Python version of the framework and there are quick start guides, there are docs. That's the best way to get started.
Speaker 2:You have an extensive list of tools that Langchain can work directly with and, similarly, if you're a JavaScript or TypeScript person, you can do the same. So Langchain JavaScript and it'll take you to the JavaScript version. If you're looking for a video-based resource, you know, as I said, I have our I think our 50-minute long course. If you just type in FreeCodeCamp introduction to Lantern in your browser, there will be a YouTube course that is available for you to take. It includes examples and stuff, and I think that's a good way to start with Lanxion. And then I have a video that's upcoming in the oh, we are in April already, so it's upcoming in this month sometime which talks about basically what we were discussing is the use of Lanxchain with Azure AI Search.
Speaker 1:I've just got a question just for myself when do you know when to use Langchain than the native Azure OpenAI, like doing with Python and stuff? How do you know when to use Langchain itself?
Speaker 2:I want to say that if you want so, like, let's say, you're at a hackathon, or if you want a really quick way to demonstrate an idea that you had in your mind and you want to build a AI LLM app, I think Lanqin is great in those scenarios. But my experience is that and this is also, like you know, working at Twilio and with other companies on different AI-powered apps has been that developers use, you know, the SDKs and the libraries directly within, like, as you said, you know, the Azure, openai or Azure Search, instead of using a framework, and I can see that why. That is the reason. So, even with langchain um, having been working with this framework for the past six, seven months, things are changingly rapid, like things are changing so rapidly that yeah the methods that you use previously don't work anymore or are deprecated and and now they have a new way of doing things.
Speaker 2:So you can't have that within a production use case, and I feel like things like SDKs that are official from Microsoft or Azure. They are more robust and have better long-term support and I think that's why there's like a mindset of, hey, if you want to build something long-term and like it's going to be used, if you're going to build something for work, yes, I would say you know build something with, like the SDKs itself directly. Yeah, yeah, that's been my experience.
Speaker 1:Maybe it'll change, um, but yeah for now okay, so that was a great question learning, learning and while sdk with like python and stuff is for like production, stuff you want to do in real world. So that's different, okay, cool. So, as this episode is coming to an end, we would love to find out about yourself. So how are you going to any events in the future tech events?
Speaker 2:oh, good question. Um, I know, um, that build is coming around. I won't be able to make it, but, um, I'm excited for you know, just the virtual experience. So Microsoft Build is coming around. I know we met at the MVP Summit, which was awesome, and just excited about all the great things that are coming within the container landscape, serverless and AI, so excited to see what all announcements we'll see at Build and what other events. So we do have meetups here at Microsoft Reactor in Toronto. I usually go speak at some of them. So there's the global AI bootcamp.
Speaker 1:That happened right.
Speaker 2:And I gave this Langchin talk and now there's global Azure bootcamp. That happened right and I gave this langchintalk and now there's global Azure bootcamp. That's happening this month, so I'm excited to attend that.
Speaker 1:Okay, that's brilliant. So one last thing. So how can others get in touch with yourself, Rishabh?
Speaker 2:Sure, so you can find me.
Speaker 1:Do you want to?
Speaker 2:know more. Yeah, so you can find me at rishabh Kumar dot com. That's the easiest way and has, like, all my links and stuff. It's RISHABKUMAR dot com. It has my Twitter, linkedin, youtube and all other social media platforms that you might use, other social media platforms that you might use, and also my GitHub and stuff.
Speaker 1:So, yeah, that's the best way to find me. Yeah, thanks. And also, you learn to cloud the program. So yeah, you can find that. Do you want to talk about more for that? For viewers?
Speaker 2:Sure yeah, yeah, yeah. So I think yeah, yeah, this is the right way. Um, learn to cloud is an open source guide that helps you learn cloud, as the name suggests. So if you are someone who is either struggling um, you know to break into cloud or are just starting out, go check it out. It it's all free. All the resources we have there are free. It roughly takes people six to 12 months to go through the entire curriculum. So it's built with first-hand experience by Gwen, who's a great friend, works at Microsoft as cloud advocate, and me, because we both are kind of like self-taught people who started in help desk and moved into cloud computing and DevOps. So, yeah, check it out. I think it's a great resource. We have around 4,000 something GitHub stars and last year we had around 160,000 users using the web platform. So go check it out. It's at. Learntocloudguide is the URL.
Speaker 1:Okay, thank you. Thanks a lot for joining this episode, richab, and in a few weeks it's going to be on social media like Apple Music, spotify and YouTube. Thanks a lot, bye.