---
type: raw-transcript
title: "Is Gemini 3 Really the Best AI Ever?"
youtube_id: YyoqKHpH1EU
url: https://www.youtube.com/watch?v=YyoqKHpH1EU
slug: is-gemini-3-really-the-best-ai-ever
published: 2025-11-24
duration: "0:12:20"
fetched_at: 2026-07-21
immutable: true
---

## Transcript

**[0:00:00]** It's the breaking news of the week. Google has just released Gemini 3. And so right now, I want to show you the benchmarks. We're going to build a landing page together in the new anti-gravity. And that's going to be all it takes to show you that Gemini 3 is the most powerful LLM that has ever been created. Just kidding. It is not that simple. And that's actually what I want to cover with you today. Because whenever these new LLMs are released, they get a ton of hype and they're

**[0:00:26]** always crushing it on the benchmarks. But then often when you try the LLM yourself for something like AI coding, you see an entirely different picture. Now, Gemini 3 is a genuinely impressive model. Don't get me wrong, we'll cover that as well. You can't deny that even if you take these benchmarks with a big grain of salt. The jumps that we have here are insane. But the problem that I want to focus on with you is that these benchmarks, they kind of seem like marketing material. And we know that

**[0:00:52]** these large language models are getting trained more and more to solve these kinds of tests and tasks. And so that leaves us with the question of like how do we actually know if an LLM is really the next big thing? And it kind of just comes down to you have to try it yourself, wait for millions of other people to try it, and then establish a kind of common opinion. Like we know that Claude Sonnet 4.5 is generally considered the best LLM for coding, at least before Gemini 3. But that kind of

**[0:01:20]** common opinion, you can't even trust that and it's far from an immediate eval. So, we're stuck looking at these benchmarks. Now, there is a solution to this that I've seen surface recently. I want to cover that with you as well. So, this video is going to be really just no fluff. We got something big in the industry right now. Let's talk about the problems. Let's talk about the solutions. All right. When we think about evals for LLMs, it helps to focus on a specific domain. So, I'm going to

**[0:01:45]** focus on AI coding, especially because of anti-gravity, Google's new AI IDE that integrates with Gemini 3. Very cool. We'll talk about that in a little bit. But here's our problem. On the left hand side, we have our benchmarks. This really nice curve here showing the apparent explosion in capabilities for our LLMs over the last months and years. But then on the right hand side, developer productivity is just a flat line. So, as these LMS have gotten more and more powerful, it hasn't actually

**[0:02:16]** translated to the real world, at least with this study here, examining top contributors to big open source projects, which in my mind is pretty legit. And my experience, I can kind of relate to this. Obviously, the results that I've been getting with AI coding assistance has completely exploded over the last year. I talk about that all the time in my agenda coding course on my YouTube channel, but it might and just might be more thanks to my tools and AI coding system than it is the underlying

**[0:02:46]** large language model. And so just as a little bit of a throwback, I was using Windsurf way back in the day. Well, I say way back in the day, but this is like the start of this year and late last year. And within Windsurf um at this time I was using Claude Sonnet 3.5 which is like four generations old at this point. So as far as LLMs go it's really really old. And so now obviously I am using Claude code with Sonnet 4.5 and the results I'm getting are better like I said. But if I were to plug in

**[0:03:17]** Sonnet 3.5 into my current tooling and AI system I wonder how close the results actually would be. I mean, even back then, Sonnet 3.5 was crushing it for me, and I was developing at lightning speed compared to using no AI coding assistance at all. And so, it's just interesting to think like how much of it is more just like our tools and systems on top of LMS making them seem more powerful when the underlying reasoning really is a flat line. Now, I'm curious

**[0:03:46]** what you think. Like, let me know in the comments what you think on this because honestly, it's hard for me to evaluate this kind of thing myself. I'd have to go all the way back to Sonnet 3.5 and test it on a ton of more complex use cases I've been working on recently. I mean, that's the problem we have where we rely on benchmarks because the alternative is just something that's really complex. Okay, so our core problem here, large language models are getting more powerful, but to what

**[0:04:09]** extent? It might be the tools we're adding on top of the LLMs that are making more of the difference. And here in Google's new anti-gravity, I have a really good example of this. And so this is their new AI IDE, obviously using Gemini 3 by default. And Gemini 3 seems to be so good at designs and building frontends. That's what I've been testing it with over the weekend here. And it seems like it's better than something like Claude Sonnet 4.5. But is it really the case or is it some tools that we

**[0:04:39]** have built into anti-gravity? One of the things that we have in anti-gravity that is very cool is a Google Chrome integration. And so what we're able to do within anti-gravity is ask the AI coding assistant to verify websites. It'll open up its own browser that it can navigate autonomously. So it can visit websites after it starts them up and it can validate things visually. It's very very powerful. So for this simple demonstration here, I just asked it to analyze my codebase so it knows

**[0:05:08]** how to start up the front end, scroll through it, and then recommend suggestions for making the website look better. So, taking advantage of the vision capabilities of Gemini 3 as well. And it worked extremely well. And not only can you have it visit sites autonomously, but you can also go through and you can watch its work. And so I can go in here, I can take a look at the screenshot that I captured on my website. I can view the playback. So I can watch it scroll through things and

**[0:05:35]** click on buttons and go between pages and things like that. It is so so cool. And so one thing that I did run into with Gemini 3 in anti-gravity is it kept getting an overload error. So I just have to keep asking it to continue. So you might encounter this yourself if you use anti-gravity. But the point that I'm trying to show here is that it has this new capability added on top of the LLM that makes it a lot better at working with frontends and designs. And I was

**[0:06:00]** super impressed with the results that I got here. Okay, so I'll show you this live right now. I'll ask it to visit this website that I already have running. I could have also asked it to spin up even the website itself. But yeah, this is just so cool. We can watch it navigate the site autonomously, which this isn't a brand new thing. You can do this with the Playright or Stage Hand MCP servers, for example. I've even shown that on my channel before, but this seems to really be optimized for

**[0:06:26]** anti-gravity specifically. And having this as a built-in integration is really, really cool. And just the way that, for example, it waits 5 seconds before it captures the screenshot to make sure the page is rendered, like that kind of thing. It's sort of like baked into the system prompt for this Google Chrome integration. Really, really cool. And so you can see it like scroll through the site there. If I go back into the IDE, I can view the screenshot that it took and the playback

**[0:06:50]** like I showed earlier. But this is all happening live now. Very, very powerful. And in general, anti-gravity is a fantastic AI IDE. They also have their agent manager mode. So it takes the code out of the picture in the IDE. So we're just working with the conversations with our coding assistants. We can kick off tasks in parallel between our different repositories. We can also view the code that it changed. We can add comments line by line, so more of a traditional

**[0:07:16]** code review, which is also really neat. So yeah, let me know in the comments if you want me to cover anti-gravity in another video. I think this is an AI IDE that is worth paying attention to. So going back to the benchmarks here, it is clear from my testing that Gemini 3 is a very powerful LLM. And even though I do take these benchmarks with a grain of salt, this test in particular, the results that Gemini 3 got here is insane because ARC AGI 2, it is a lot more than

**[0:07:43]** just pattern recognition. It's giving tasks to the LLM that requires a lot of abstract reasoning. And so, yeah, Gemini 3 is legit. But like I said, the problem that I have here is when I'm testing it myself, there's a lot of tools and systems built on top. And so even after this weekend, I really don't feel like I have a super solidified idea in my mind of how powerful Gemini 3 actually is. And so that's where we get into the solution that I want to introduce here.

**[0:08:13]** The higher level idea and then one in particular with Kleinbench. Think about this with me for a second. If only there was a massive set of publicly available repositories where you could see how they change over time as real engineers are working on real tasks with AI coding assistants. Oh wait, that's exactly what we have with open source repos, especially with something like GitHub. And that is what Klein is going to be leveraging with Kleinbench. And now I'm

**[0:08:40]** not affiliated with him at all, but I really do think that they're on the frontier now of LLM evals, especially as it relates to agentic coding. And the problem they go over at the start of this article is exactly what I've been talking about today. Our benchmarks just don't resemble how we are actually using large language models in the wild. And so for coding specifically, it's all just a bunch of leak code style puzzles like reversing a linked list for example. And I know this is AI coding

**[0:09:06]** specifically, but you can extrapolate this like I've been saying to any benchmark. And so what client bench is doing for us, there's three primary purposes. First is reliable evaluation. So instead of relying on these synthetic benchmarks, we are now working on real engineering tasks. And so when you opt in to use client for this, it's privacy first. You have to opt in. Obviously, it will track your prompts and how you're working on these real engineering tasks with client on your open source

**[0:09:33]** repositories so that we have real engineering tasks for our eval. And then also by standardizing and publishing these environments like we obviously have to work in controlled environments to make these eval reliable. It's also a way for us to document system gaps and so we can learn how to use AI coding assistance better as we are creating these eval. And then also this is all going to be training data for research and fine-tuning down the line. And I know that training data is always a

**[0:10:00]** scary thing for privacy but they have this at the top of their mind. So they talk about that as well. This is something that you have to opt into and is only available for open-source repositories. And client bench has designed things very very intelligently. There's just three pieces of information that we need for any real engineering task. We need the starting snapshot of the repository, their prompt, and then also the end state or the code that was actually committed to the open- source

**[0:10:24]** repository. Now, I hope that starting prompt can also be extended to many different prompts if you have a whole system you run before you make your commit. But yeah, the point here is that it's really simple. We just need the starting state, the process, and the ending state. That's all we need for our eval. Now, it's kind of simple just looking at it here, but really under the hood, there's a lot of challenges that client is going to have to address because like we've been talking about,

**[0:10:49]** different engineers are going to be using different tools and systems as they work with their AI coding assistance. And so, we have to account for that because maybe one engineer gets a lot better results with Gemini 3 just because they have more of a process than someone that's using Cloud Sonnet 4.5. But as long as that is documented in the process here and they have some kind of standard for these agentic engineering environments like they talked about up

**[0:11:12]** here. As long as we have that standard then this is going to be so powerful giving us real world data understanding the LLM that we should use for our own projects. And that's the whole point of this video is to show you that like this is the direction we're heading to make it so that you know better the tools and LLM to use for the things that you want to create. And one thing I want to be really clear on here is that clientbench might not end up being the standard for

**[0:11:35]** AI coding eval. The point is not the individual company or tool, just that we have to move to real tasks for evaluating our LLMs for things like AI coding. And that's everything that I've got for you today. One thing I want to plug really quickly is that I have a super exciting live stream happening November 29th at 9:00 a.m. Central time. I'm going to be giving away my remote agentic coding system that I've been working really hard on recently, but I'm only giving it away during the live

**[0:12:03]** stream itself. So, come be a part of it. It is going to be an absolute blast. And so, with that, if you appreciate this video and you're looking forward to more things on AI coding, I'd really appreciate a like and a subscribe. And with that, I will see you on the live stream on November 29th.
