Speaker 1: 0:07

Hello, Welcome to Microsoft Community Insights Podcast, where we share insights from community experts to stay up to date in Microsoft. I am Nicholas. I'll be hosted in this podcast. We'll dive into Azure AI Foundry, but before we get started, I want to remind you to follow us on social media so you never miss an episode, To help us reach more amazing people like yourself. Today we have a special guest called Rufi Watsi. Can you please introduce yourself please?

Speaker 2: 0:34

Yeah, hi. Hello everyone, this is Rufi Watsi. I am happy to be here talking to Nicholas about the Azure AI Foundry and new services that are into it, and I've been working with Microsoft for almost three years now and I'm part of Azure Core Partner Ecosystem team, where I interact with a lot of partners understanding what their platform requirements are and how could we make it a better platform experience for them.

Speaker 1: 1:02

Yeah, so before we dive into the theme, ai Founded, can you share how your career get started with Microsoft and how you got to AI?

Speaker 2: 1:14

Yeah, sure. So before joining Microsoft, I have been working with a lot of partners in general, but also I had done a good stint with Accenture for almost 11 years and started as an application developer and then did a lot of management work in terms of understanding requirements, customers, partners altogether, and then creating that experience of holistic solutioning and design fundamentals. And from there now, currently, when I joined Microsoft, I started as a TPM on modernization practice for specialized workloads, for mainframe and mid-range workloads. It was really exciting to see how Microsoft is doing that for big, mission-critical customers and then moved on to this current role for AI partners where, like I mentioned, we work with partners so that to understand their platform requirements but, at the same time, also understanding what are the blockers that are preventing these partners or if there is anything better that we could do to make their experience even more smoother. So it could be related to storage, network or compute related.

Speaker 1: 2:28

So majority of the job is just gathering feedback and listening to customers and work with the product team. Exactly Okay. Before we dive into the theme, what does it mean by TPM? Just for viewers?

Speaker 2: 2:46

Yeah, so the TPM is the Technical Program Manager, and a big part of your work on a daily basis would be to drive clarity and remove ambiguity, understanding what, and you have to look at the program level but also stay deeply technical, where you could formulate the ambiguous requirements into features, and those features are the requirements that you have distilled working with various stakeholders, and then you take it forward with your product team and make things happen.

Speaker 1: 3:21

Okay, so, as we know that AF Foundry was first introduced in Microsoft Ignite last November and we just want to find out more about the missions and what it is, can you?

Speaker 2: 3:36

tell us more about it stop solution for developing or improving customer AI apps or solutions, and it comes with all the goodness of security embedded security, embedded trustworthy controls and features that enable you or your developers to create that experience end-to-end. So it has many elements to it. So, in terms of understanding what kind of model I do need, we have 1800 plus models available in our model catalog and, based on our requirements, you could filter it out. Maybe it's for classification, maybe it's for summarization, or maybe you want to fine tune your model so you could select from those specific needs. And then it also have other embedded features. Like you could do evaluation based on the instructions that you are providing.

Speaker 2: 4:31

You could manually evaluate or have some database evaluations too, or you could do fine tuning, as I mentioned earlier. In nutshell, it's a holistic experience for your one-stop solution to come and elaborate on the solutions that you already have and test your solutions, deploy them, and the portal experience is pretty intuitive. So, even if you are a beginner and you're just trying things out, I love the playground experience that we have, because a lot of time we don't know what exactly we would be expecting and which model we want. We also provide a lot of comparisons between models so you can come to the model catalog and see for yourself what are the kind of parameters you want to you want to test your models against. So all that benchmarking is already done in place for you to explore because I noticed that the Mongo catalog have lots of model.

Speaker 1: 5:27

So how do we know what model is good at what use, what use case?

Speaker 2: 5:32

That's a great question I was mentioning. In the model catalog we have classification based on your needs. So, for example and it ranges from different kind of scenarios you might have so we have collections to start with that. In case you want to go with a certain model provider, you could select those. But let's say you want to keep your options broad but have specific industry needs, so you could select which industry in particular are you interested in. But then comes this is my favorite part about capabilities that do you want something supported by agent or do you want something supported by interest? Now let's say you are not clear about which collection you want, maybe not very specific about industry, but in general you want to test things out.

Speaker 2: 6:21

Then here comes the inference task. So you could select from a wide range of text to speech, embedding classification both for text and image. So let's say we look for summarization. So the good part is that it comes with all the models that are available in the catalog for summarization and then you can further distill it. Hey, I need only models from, let's say, microsoft, as well as maybe Meta or Ristra. So in that case you could get a collection that you will be looking at.

Speaker 2: 6:54

But another important thing to mention here is that there is fine tuning tasks too for all these tasks that we talked about, for inferencing as well.

Speaker 2: 7:03

But if at all, you're still not sure and you would like to see how things would be when you would, let's say, be in terms of benchmarks, you can go to the model and then see what was the data it was trained on, what kind of inputs was it, what kind of, and it sometimes gives you a lot of other data sources that it was trained on.

Speaker 2: 7:25

So you're getting really into the all the weeds of what fundamentally went into building and training that model, and then you could compare with benchmarks, like I mentioned earlier. So, let's say, you want to compare a certain model or series of models. So, for example, here I have selected these five models. Now, for all of these five models, I can get these particular metrics to compare and I can build my own metrics as well. I mean I can build my own metrics as well. I mean I can build my own view to compare. So if I want to change my x-axis from quality to cost and my y-axis to something else, then I could play around with that as well and I could add and remove models based on my needs, so that is pretty cool as well.

Speaker 1: 8:14

That is pretty cool as well. Yeah, so I think a few days ago yesterday, I mean the Azure AI agent service has been released public preview. I think you kind of want to show us what it is and go into more detail.

Speaker 2: 8:26

Absolutely, absolutely. So let's head into the agent's view and right now, if you go into Azure AI Foundry under build and customize, you see the agent preview. You head into that. So let's go right into it, and I have been trying myself since yesterday, which is I love it because it has a lot of cool features. But let's go through the features one by one. So, yes, you can definitely create new agents, but let's go to the one that I already have.

Speaker 2: 8:56

So while I was setting this one up, I had to give my own agent name, the resource connection and the model that I would like to deploy. So I have selected GPT-4-O-mini and you could like a system message. You could give it clear instructions on what needs to be done. So let's try it out and let's say what do we want from our particular agent to do? In this case, I want my agent to look at some of the data books related information that I have in my document and I wanted to refer to it. So, let's say, are a book assistant and help people with book recommendations and related questions. Let's start with that and then we can build on our system message as well. But to do that because I wanted to specifically talk about my document. I want to add the knowledge source and here you could try using files, azure, ai search or grounding with Bing search. So I'm using my own file search and I have a document. So I'm using a select existing vector store and with that I have got my data, which is my book's details. I upload and save it and there I have it. It also has actions, so you could define actions through logic, app functions or code interpretive. So those capabilities are also there For this particular demo.

Speaker 2: 10:25

I am not exploring those options, but definitely you should try those as well. And again, it comes up with the settings. So if you want to have more innovative and non-deterministic responses, keep the temperature high, and there are other settings you could play with as well. And if you want to turn to playground right from here, then you could directly redirect to playground. And here we are, in the playground. The playground experience is very similar to what we have for non-agent chat or other speech kind of playground experience. It doesn't change, change drastically. That's why I find it pretty intuitive, because we already have experienced that through our regular playground experience.

Speaker 2: 11:10

So here let's try it out, let's give it some options and start with hi and maybe ask questions on a specific book that I have related to Britney Spears. So let me say what is the name of Britney's book, and I'm not going to give it the full name. Let's see if it's able to tell. So it is exactly looking into my document because that's why we are having that document in place, so that it refers to the document for any information. Now it's telling me the name of the book. It tells a little bit about the book. Let's say I want to know the price of the book. Can you put the book's price and ratings and availability in our table and let's see what it does there? Although it shouldn't spelling mistakes shouldn't matter, let's see what happens here. Okay, there we have it. So we have the title, we have the price, the ratings and availability.

Speaker 2: 12:18

Now I have done a single agent, but you could do multi-agents as well, using semantic kernel and autogen, and orchestrate your agents into a more streamlined and in a more architected pattern. So you could have an orchestrator agent which is going to define, based on the call or the request, which specific agent expert it should redirect to. So consider this as a system of multiple agents. Every agent is like a microservice. It's specialized in its own thing and, based on a request, the orchestrator is able to decide which particular task can be best fulfilled by which agent. So there you have it.

Speaker 1: 13:01

Yeah, it's quite simple to set up your own agent service. It is so. Is there any particular use case where, specifically your case, where AF Boundary is strong against, has the most impact to customers?

Speaker 2: 13:16

Yeah, I think it's all about creativity and innovation at this point, because that's the whole idea. You try it, you try it, you build it for your own cases. To me, there is no limitation of the use cases. That could be best served by the azure ai foundry and it's very developer friendly. Also, very quickly, you could go to the sdks and define your own tooling. So when you're building agents using SDKs, we have a concept of creating a tool set so you could create your own tool. So the tools that you saw on the portal on the right-hand side, that you could create a portal for Bing search or your own files or file search all of that becomes tools when you are in the SDK and then you can plug your own tools and then create the whole tool set about it and then invoke it in the tool and then the agent is going to decide, based on the queries, which particular prompt will relate to which tool set being invoked.

Speaker 1: 14:18

So, based on your experience of using AI or working in AI in your time at Microsoft, what are some of the lessons learned or highlight that you think organizations should consider when adopting to AI?

Speaker 2: 14:32

I think first is to start. I think we all have experienced this and me myself that it's must to start first and then see how you would scale on it. Never try to replicate what others are doing, because a lot of times you spend a lot of money and don't get value out of it. So I would say, start small and then build upon it. And if you have a very clear view on what needs to be done and AI seems to be the best choice to do it, then definitely that's the right program. But even if you're not sure about how you want to leverage AI, I would definitely recommend going through our use cases and some of the industrial models that we have created in collaboration with other partners. For example, just yesterday, fujitsu explained how they have been using AI sales agent for enhancing the productivity of their workforce by 70%. So you could use those examples also to understand how the industry or other people are using AI. But definitely that would be my best recommendation.

Speaker 1: 15:49

Is there any particular resources that people should be aware of when trying to learn AI Foundry?

Speaker 2: 15:56

Absolutely, you should head into Microsoft Learn and we have a lot of documentation on Azure AI Foundry, both for portal and for SDK should definitely check it out. It has a lot of details around what kind of nuances are there, what kind of FAQs people might have. So it is pretty well vetted and considering all the experiences that people might want from the portal, so please give it a check.

Speaker 1: 16:24

And also I want to add a note, you can also find some like resources through the max of reactors as well, because occasionally do some live sessions for you to learn more about ai, foundry or ai in general. So as we come into the end of this episode, we always I always want to get to know our speakers. So what do you normally do aside from your work at microsoft? Do you have any hobbies or interests?

Speaker 2: 16:49

I love water, so I am now learning I'm really swimming. I used to do a lot of swimming, but then I lost the touch, so I'm relearning swimming. So that's something that I am focusing on right now, and I'm also focusing on tennis lawn tennis. So, again it it is, I'm getting there, but not there yet. So those are two things. Besides being outdoors, I'm developing the habit of book reading and I am reading. Recently, I read a book about how to feel empowered with by being yourself. So that's something.

Speaker 1: 17:26

So that's something that I'm reading right now you can probably start like a book club internally I would love to, but I don't think I am there yet.

Speaker 2: 17:36

I've just started certain things about developing the habit of reading every day is something that you're working on.

Speaker 1: 17:43

I know that swimming is quite hard because sometimes, when you forget it, you have to start from the basic, so it's quite hard. Yes, yes. So what's your plan now? Are you going to any partner events or anything in the near future?

Speaker 2: 17:58

I don't think I have anything planned. Definitely, I look forward to meeting you at the mvp summit, so that is something that I'm very excited about, but outside of that, nothing as of now yeah, and also, do you have a mac?

Speaker 1: 18:13

is? Our bill is in may, so you have that plan?

Speaker 2: 18:16

yes, yes that is something that I'm still working on, so let's see, yeah, what cool ai partner stuff we could bring in there yep.

Speaker 1: 18:24

Thanks a lot for joining this episode, privy, so it's a great to chat with you and learn more about ai foundry and get some insights about the ai service, because it's quite easy to to learn. It's just very quick to learn about the ai service. Uh, yep, so in a few weeks it's's gonna be on social media, so thank you, thank you much take care.