Gemma 4 Makes Local AI Finally Possible?

Google dropped Gemma 4, built from the same tech as Gemini 3 under Apache 2, so it's it's free to use. Command AI There are four sizes, apparently. There's 2 billion, 4 billion, 26 billion mixture experts, sorry, and a 31 billion, obviously big daddy. Um so a range of different um sizes and all that lot for your needs. So, what are we looking at here? It's Quen 3.5. This is the ELO score. A measurement they're using, but we see that the it's significantly better than Quen. Yeah, and then there's also the size, right? Yeah. Well, you got Quen This is um this is 27 billion, and then this is uh 397 billion. So, the size there the it is beating Quen because in comparison to its size. So, these are 26 billion, that's 27 billion, 31 billion just a little bit ahead, and they're all roughly um you know, well, I mean, this is amazing. This is where you want to be, right? So, really really cool. Uh GLM 5 This it is our good friends MiniMax M2.7 then, no. Uh but, you know, GLM 5.1's just been released, so it's a shame we don't get to see that one. Built from the first uh same technology as Gemma 3. Uh Gemini 3, not Gemma 3, correction there. Uh most people model family you can run on your own hardware. So, I wonder what the actual I mean, let's open this up over here. And do we see a actual like file size? Uh files and versions here. And 10 gig by the looks of things, I reckon. I reckon around 10 gig. That's the smallest one, so it's not tiny tiny. I would expect something around 2 billion to be half that size, to be honest, but it can do a lot for uh for just 2 billion parameters. The entire family moves beyond simple chat to handle complex logic and agentic workflows. So, this could be a interesting you know, vibe coding tool. I might have to try this, to be honest, try and see if I can get it working on my vibe coding setup. Powerful, accessible, open to power the next generation of pioneering research and products. We've sized sized the Gemma 4 model specifically to run and fine-tune efficiently on hardware, from billions of Android devices worldwide to not not uh not Apple. Um to laptop GPUs all the way to developer workstations. So, yeah, I think I think I'll be keen to put this to the test, definitely. Cuz you we we've got all these Chinese models that are making a lot of waves like yeah, Kibi Kibi K 2.5 um MiniMax um none of none of which can really run on your hardware. It was just Quen 3.5 which we did a we did a little thing on a few weeks ago which could actually run on smaller hardware, um which was okay. I see this yet as another win for Google, uh especially now Google is again picking up steam as we see. Um something that we didn't cover, by the way, another reason why you absolutely need to watch the show Silicon Valley. It's like still relevant. Um we didn't cover this Google research introducing TurboQuant, our new comparison uh comparison algorithm that reduces LLM key-value cache memory by at least six times and delivers up to eight times the speedup. I'm not sure if this is directly contributing to Gemma 4 or not, because this is about the the memory cache. But this is another thing that is Google doing for like reducing uh and just making the the model smaller and faster. Um which is even more important if you know I don't know if you know about uh the the new limitations by Claude. So, Claude now uh has introduced more limitations uh during peak hours to users. And it just shows that it is It's very new. It's very new. I think it came out a few days ago that they are introducing new limitations, so you hit your limits much faster. Why all this matters is because it seems like we're not getting the models much more powerful, at least not order of magnitude more powerful. What we are getting is a lot of uh efficiency gains and making the models smaller and making the smaller models uh faster and cheaper and things like that. So, this becomes even more important, so we can do maybe swarm of uh swarm of agents uh and that kind of things. If I also read here uh this tweet also resonated. Uh it cannot be overstated how big this news is, being able to run a frontier AI model on your own hardware means token costs are effectively free. Given that you have the computer and the electricity, but yeah, it is effectively free. AI will be uh you I don't know how to pronounce this. Ubiquitous and cheap ubiquitous and cheap as the internet itself. This is This is like really big. It is becoming uh a commodity like a truly in a way that you can have proper models on your own hardware and for all sakes of um uh privacy and things like this. Uh this is a dagger in OpenAI and Anthropic. Nick, I don't think Nick has used this model. You know, what he's saying in theory is correct, right? We're definitely moving in that direction. It's the direction people a lot of people want. But like I don't know if you know this, but my channel has been really peaking testing local models, running local models, and using them to code, using them with open claw, doing things with them. And yes, granted, my uh usage is primarily for code focus. I would say that's one of the least demanding things outside of chat that you can do with AI. Running this on an M5, you know, again a MacBook Air, but I don't think we're going to see leaps and bounds when it comes to the Mac Pro and the Mac uh sorry, so M5 Pro and the M5 Max. I I still don't think we're there with AI. And what he's saying here seems like sounds like that he's actually run this and it's actually something usable, but but I've got a hunch that he hasn't. So, I put Basically, what I'm saying is the long short of it is I want to use this. I want to actually see it in real-world practice, cuz this sounds like it could apply to anyone, but I reckon what he's saying applies to those few with very very expensive hardware um just being able to run it locally, you know? So, yeah. I'm apprehensive to take [clears throat] his tweet to the everyday person like you and I with our MacBooks and our M1 Pros and our, you know, whatever. I I agree. I personally tend to use the the models in in their absolute maximum setting, unless I definitely think that the task shouldn't be over think. Um if it's the case, I will drop to a lower model, but I don't mind um waiting 10 extra seconds or even a minute to get something better. I don't even use the Haiku. So, I I don't want I don't know where this is standing comparison to Haiku. That that's [snorts] uh what would be interesting to me. Uh at the end of the day this this is this is an open model. Um you can run it on your own chip. May maybe there are people with like lightweight enough tasks where they can use this. I don't know. May maybe there are tasks like that. Um maybe it is going to be useful for bigger models. Uh maybe you can have Sonnet creating tasks for [snorts] your local um AI. That could be something in the future. Um yeah. But herein lies the problem though that um means that people will will never be happy with AI locally right now, and that you're comparing it to and many other people will be comparing it to the frontier models, OpenAI. Yeah. Like you will just not get that. People were surprised that hang on, how are they model that's running on um hardware that's halfway across the world? How is that still faster than AI running on your actual machine? And the truth of the matter is they've just got such powerful hardware. You need such powerful hardware to run this stuff. And this when people first start getting into AI, they're like they are comparing it to the frontier models, and they are weighing up those options. And ultimately, if you are doing that, you're going to be very, very disappointed with how well AI is running locally for the everyday person right now. Quen 3.5 was a glimmer of that running on that app, but that's just doing chat. And who really cares about AI chat running locally? I mean, it's nice, but people want to get real work done, and I just I just don't think we're there yet. And all that to be said, I want to try it. I want to I want pull this down and give it a go. Definitely. It is quite fun. This is part of a larger conversation on my show, Command AI, which we stream live every single week. We discuss the news and all things related to AI in the world of design and web. Catch us next week and join in the banter. Command AI [screaming]