---
type: raw-transcript
title: "Build a RAG AI Agent with REAL-TIME Source Validation (CopilotKit + Pydantic AI)"
youtube_id: Be2OQ3LQZcQ
url: https://www.youtube.com/watch?v=Be2OQ3LQZcQ
slug: build-a-rag-ai-agent-with-real-time-source-validation-copilotkit-pydantic-ai
published: 2025-12-21
duration: "0:17:04"
fetched_at: 2026-07-21
immutable: true
---

## Transcript

**[0:00:00]** One of the biggest problems with most rag systems is that we don't actually know the sources that our AI agent is using. Did it hallucinate? Did it pull from irrelevant documents? And no, citation is not enough because agents can make that up as well. And so today, I want to show you how to solve this problem with a rag agent that you're looking at right here that has human in the loop for source validation. Users can see the chunks that are returned from our knowledge base, approve some

**[0:00:30]** and reject others, and only then does the agent synthesize an answer. So, we have full control and visibility. And boy, is this powerful. So, I built this agent with Copilot Kit and Pideantic AI. I'll show you what that looks like under the hood. But first, I want to give you a quick demo of the agent. So, the knowledge base that this agent has access to just contains a bunch of information about different AI companies. And so I'm going to restart the conversation here and paste in a

**[0:00:57]** question. I know that my face is blocking this a little bit, but it's just asking what is OpenAI's latest funding. So I'll send this in. And before we get an answer, we're going to see the chunks appear on the lefth hand side. And the agent recognizes like okay, select the sources you want me to use for the summary and then ask me to proceed. And so I can check the box for all the chunks that I actually want it to include. And we can also click into this to see exactly what we did pull

**[0:01:21]** from the knowledge base. There's so much transparency that we have here. Even using this as an application just to debug our rag agents and evolve our rag agents is super powerful. So I select a couple of chunks and I can even say what are the sources that I have selected. And so I'll show you that we have this real time sync between the agent in the backend and our front end. It knows the three sources that we have selected. All the other ones right now we do not have

**[0:01:49]** as approved chunks. And so now I can say yes, give me the answer. And so we'll get the final answer, but it's based on everything that we have curated for the agent. This is not the kind of thing that I've shown on my channel before. And also, this is not the kind of thing that is easy to engineer. Having this real time sync with our actions in the front end and what the agent is working with in the back end is very powerful. Now, as far as use cases for this,

**[0:02:14]** something that I've been doing a lot now is treating this as a rag agent harness. And so essentially I'll take any rag agent that I'm currently working on and I'll plug it into this front end so I can test its responses with different sources selected and then also just make sure that it's pulling the right knowledge from the database. And you could definitely adapt this to be more of like an internal rag agent where users maybe like on your team can actually like pick and choose the

**[0:02:39]** sources that the agent is searching through. So treating the chunks return as a recommendation that you then get to sift through yourself if you want to make sure that you get the most accurate responses possible. Now for the two core components of our tech stack that makes this rag agent possible. We have copilot kit for the front end and pyantic AI for our backend agent. Copilot kit is what enables agentic experiences as I like to call it. So there's a video where I've

**[0:03:04]** covered this on my channel already that I'll link to right here with aentic experiences. They also call it generative UI. It allows our AI agent to manipulate the state in the entire user interface. And then also we can change the state of the agent from the UI. So we're just really deeply embedding the agent in our application. And I really do think that this is the future of SAS and really all web development. And the C-pilot kit team also built the AGUI protocol. This is the secret sauce that

**[0:03:35]** works under the hood to connect any application to any AI agent. And so in our case, we have a copilot kit app that is connected to a paidantic AI agent. And pretty much any agent framework you could ever hope to use supports agui like langraph, crewi, mra, agno. And each one of these in their docs, they'll tell you how to do that integration. And so, for example, with Pantai, they have the dedicated page here that I'll link to in the description, of course. And it

**[0:04:02]** shows you how easy it is to take any Pantai agent and instantly make it available as an API endpoint that is AGUI compatible. So, you can connect it to your C-pilot kit app. And now we get to the fun part. How we connect this all together because we talked about Copilot Kit and Pideantki and Age UI. How does this all come together? And that is with the use agent hook which really is the star of the show. This was released by the co-pilot kit team just this month.

**[0:04:30]** And what it does here is it gives us this agent object. And so we have direct access to the agent in our front end but the agent is still running securely in a backend API. And so now we can manage the state for the agent. We get real-time updates managing the conversation history all from the front end. And because it's using AGUI under the hood, we can connect with any agent, no matter how you built it or the framework that you use. And as they put it, we can access and interact with our

**[0:04:57]** direct to LLM agents directly from React components. And let me tell you, this is something I always wish that I had. I mean, yes, we can build this kind of real-time syncing solution with our agents manually, but it is a fundamentally difficult problem to engineer. And I think you'll see that as I get into the rag agent with you. The problem that we're solving is not easy, but our solution is pretty simple because we are using the use agent hook. And so it gives us direct access to all

**[0:05:23]** the information about our agent even like if it's running or not. So we can have real-time updates as we are waiting for a response from the agent. We can list out the conversation history and manage that in our front end. Working with the underlying state that it's using for things like its tool calls, even updating the state for the agent so the next time it uses a tool call, the state is changed. like this kind of thing you have to do through a lot of different API endpoints and calls if

**[0:05:47]** you're not using Copilot kit and AGUI even subscribing to agent events and so just making your application more ingrained with the agent and making it feel a lot snappier because you're constantly getting that responsiveness as you're interacting with the agent. All right, so now that we know all the components in play, I want to get into our rag agent specifically and how it works. And of course, this is a fully open- source GitHub repository. And so I'll have a link to it in the

**[0:06:14]** description if you want to try out this agent for yourself. Everything is completely free, obviously, except for the large language model that you have to hook into the agent in the back end. And so, let me actually restart the conversation here and ask a new question. Tell me about Amazon's investment in Enthropic. So, I want to showcase the features again really quickly and then get into how everything works with the use agent hook and our Pantic AI agent. And so first of all

**[0:06:38]** what we can do here is we can select different sources. We can deselect them or select all. And as I'm making these changes here it is being synced in real time to our agent in the back end. And so for example I'll select just one chunk here. I'll say you know what source do I have selected. And so it'll say you have chunk one cloud AI wars. There we go. All right. So it knows right away what I'm doing in the front end. And so now I can say okay great. now give me a summary based on that. So

**[0:07:09]** I'll send in this request and then what we can see if I go to the backend logs in the API we can see that for approved chunk ids we just have a single chunk. So it's chunk index number one for the document titled cloud AI wars and so right here AWS has made a significant $8 billion investment in anthropic. I wish they would invest $8 billion in me but yeah there we go. That is our answer based off of just this piece of information. And we can even click into

**[0:07:36]** this to see exactly where we got that. There we go. $8 billion investment from Amazon Web Services in Anthropic. And if you were to select the chunk that doesn't have that information, it would just straight up tell us because it is only leveraging what we have approved here in order to synthesize its answer. And so think of it as like these are our recommendations and then we get to curate what it actually uses. And of course, I have a nifty Excal diagram to

**[0:08:02]** help you understand how this all works at a high level. And then I'll get into the weeds a little bit more in the code in just a little bit. But I do want to stay relatively high level just talking about how everything works together. And so first of all, we have our backend agent. And so this is a pideantic AI tool to search our knowledge base. If you've done anything with Pyantic AI before, you know how dependencies work. These are the runtime dependencies that

**[0:08:27]** we inject into our agent for all of the tool calls and the state. This is what we're real time syncing with our front end using agui. This is available as just another one of the dependencies for our agent. And so we send in the first query to ask the agent to search the knowledge base and it's going to run this tool and the tool returns a state snapshot event. This is the event that is streamed to the front end. So the front end knows what chunks to display to the user to get to that approval

**[0:08:56]** stage. And so now in our front end, of course, we have the use agent hook, the star of the show. So this is connected to our rag agent using copilot kit pointing to our API endpoint for the pyanti agent. And then whenever we call this rag tool, we return that state snapshot event. And so now we have the updated state for the retrieved chunks for us to display in a react component. And then when the user approves the sources in the UI, we're going to call the set state here on the agent and

**[0:09:26]** we're going to update the approved chunk ids. And this is instantly going to be available now in the state for the agent in the back end. And so at this point, the agent has just offered the sources to approve or reject. But we are blocking the agent from giving the final response until the user does the approval and then sends the next message to the agent like we saw. And once the user does that, then we go back to the back end. And there's another tool that the agent has to synthesize the

**[0:09:52]** response. And so we're going to look at the state. This is what is updated from our front end. We're going to fetch the approved chunk IDs. And then we're going to generate the answer based on only the chunks that we have approved. And so that way we have human in the loop rag. And that honestly just unlocks a whole new level of reliability for our agent. Now, I also want to spend at least a little bit of time diving into the code with you because I want you to

**[0:10:17]** understand how easy it is to take our use agent hook with copilot kit, our pyantic AI agent, and connect them together with AGUI. And of course, I encourage you to take your AI coding assistant and ask it questions over this codebase if you want to understand more how it works or especially adapt it to your own use cases. And I do have instructions in the readme here for how you can get it all up and running yourself if you want to try it out. So the first thing that is really important

**[0:10:44]** for you to understand here is how the state works. And so these are all of the pieces of information that we want to sync in real time with our front end and backend. And what you'll notice here is that all of the types defined in our backend. This Python script right here is exactly the same as the types that we have defined in the front end. And this is important because the state is really a contract that we have between the front end and back end with AGUI. So the

**[0:11:10]** only reason that this integration works so seamlessly is because the front end and the backend understand the state in exactly the same way. Same structure, same types, same everything. And so this state is the dependencies for our pyantic AI agent. And so we define our Panti agent just like I've done a million times on my channel already. We get the large language model and I support Gemini, OpenAI, Open Router, pretty much everything here. We get our system prompt that I have from a

**[0:11:39]** separate file. And then we define our dependencies. And this is exactly what we have through AGUI syncing with our front end. And then for the first tool in our workflow, we have the one to search the knowledge base. And so the tools are defined very similar to what you've done with Panti in the past if you've worked with it before. But when we access the state here from our dependencies, this is what we have from agui. This is the real time state sync with our front end. So it instantly has

**[0:12:06]** access to anything that we've been doing in the front end where we use the use agent hook to update the state for our agent. And so I don't need to get into the code for this tool too much, but we have both hybrid and regular semantic search. And the important thing here is this tool is not going to give the chunks for the agent to give its final answer. Remember, it's just going to list out the chunks and then send this state snapshot event so that the front

**[0:12:30]** end can take all this, display it for the user to then approve and reject the different chunks that we have here. And so the system prompt for the agent also describes this workflow as well. Like when you first get a request from the user, you call this tool and then you display the chunks. You don't give the final answer yet. We do display the final answer for this bottom function right here. So synthesize with sources. Once the user has approved the sources,

**[0:12:56]** then we're going to fetch that from the state. We're going to get the list of approved sources here. We're going to build up this context to then return it to the agent. So now the agent is going to take this information to give us the final answer. And for the icing on the cake here, take a look at how easy it is to take our Pantai agent and make it AGUI compatible. We just have to call this function and then it gives us this app instance that we can import within

**[0:13:22]** our main.py where we are setting up our fast API endpoint. And so I just have to mount the AGUI app and then boom, our Panti agent is immediately available on port 8000 thanks to AGUI and I'll show you how we set this up the connection here in our front end. And so first of all in our layout we are wrapping our entire React application with our co-pilot kit runtime. This is what makes it so that when we chat with our agent in the front end, it is automatically

**[0:13:49]** going to route the request to this API. And this is where we create our copilot kit runtime to make that connection to localhostport 8000. That is where we are running our paidantic AI agent. And so now within the page.tsx, this is the main part of our React app. There's actually not that much code here. Overall, think about how many API calls you'd have to make for every single little state sync and calls to the agent. We don't have to do any of that thanks to our use agent hook. Like

**[0:14:17]** literally all we have to do is call use agent. We give it the ID of the agent that we want to connect with. And then that gives us the instance of the agent that we can interact with in any React component however we want. We can read the state and work with that any component. We can set the state and that is automatically reflected immediately in our backend. Like this is our function here to handle when the user checks a box to approve a specific chunk. And then we also have some

**[0:14:42]** configuration which I haven't shown yet but we can change different attributes for how the agent performs rag under the hood like semantic versus hybrid search. We can reset the conversation. We can select or deselect all sources. And so all of these would have to be individual API calls that we make to sync the state or interact with the agent. But now we don't have to do any of that. It's all handled for us thanks to the use agent hook. That's the power that we have

**[0:15:07]** here. here. Like if I didn't have the use agent hook, this would be hundreds, maybe even thousands of lines of code interacting with the different endpoints to handle different parts of updating the state for the agent. Like I don't want to engineer for that. So I'm very thankful that we have copilot kit and agui making this possible. And also like I mentioned earlier, copilot kit recently had their big version 1 release that came with the use agent hook. Now, the use agent hook is probably the

**[0:15:34]** biggest thing in this update, but there are a lot of other really incredible things as well. So, definitely check this out if you're interested in building these agentic experiences, applications where the agent is deeply integrated with the front end. I do this kind of thing all the time myself. I love Copilot Kit. I've got a direct line of communication with the team because I appreciate the platform so much. And of course, I worked with them on this video as well, getting their feedback on

**[0:15:58]** things. So, thank you to them for that. Now, the other thing I want to call out really quickly for the 1.50 release is their support for A2 UI. So, this is Google's generative UI spec. Essentially, all a generative UI spec is is giving agents or LLMs the ability to generate front-end components dynamically. Hopefully, that's pretty clear why that's such a big deal for CopilotKit and AGUI. And so, you can literally make it so that your agents can generate dynamic user interfaces

**[0:16:27]** based on the conversation. And that is very exciting. And so I just want to end by saying, let me know in the comments what other kinds of use cases you'd want me to build out with C-Pilot kit and pineanti to create more agentic experiences or maybe even building out this rag agent more too. I really do want to turn this into sort of like a rag testing harness. I think that'd be really powerful as an open- source project. So curious what you think. So with that, if you appreciate this video

**[0:16:54]** and you're looking forward to more things on building AI agents, I would really appreciate a like and a subscribe and I will see you in the next video.
