The Worlds Best Open Source Model Finally Released its Weights

The world's best open-source model going toe-to-toe with frontier models like Opus, like GBT 5.6 finally released its weights. Command AI. We've already covered Kimmy K3. We don't need to talk about anymore. But what does this mean? What does the open releasing the open weights mean? Well, first of all, it means that we get all of these providers able to provide the model for different prices. Now, if we look where is it here? Fireworks fast is slightly more expensive about like you know 2/3 more expensive but the speed and latency is a lot better. So the tokens per second it generates a lot faster and the actual latency in between calls is a lot faster. So they can afford to charge a lot more. One thing I did want to call out is you're paying $3 and uh $3 in $15 out for open uh digital ocean but the latency is pretty poor and the tokens per second is pretty poor. Why would anyone mention that just lotion? I don't know. I I wanted to ask like nine tokens per second. That's that's really low. What that also means is if we go on to Well, let's go to Unsloth here. They managed to run quantise the model down to one single bit which means they shrunk it from 1.56 terb to 594 GB which means it can run on a Mac studio with 512 gig of RAM plus 125 gig of RAM device. So still very much out of the wheelhouse of many many many people. However, this is pretty uh let's say um obtainable if you are if you're willing to to commit to it. What is the token per second for that? We don't know. I don't know the token per second. Just for comparison, uh GPT 5.6 soul uh achieves up to 750 tokens per second. Um that's not what we get. we get probably closer to just 50 just just to have a comparison. Well, to be fair, it will obviously depend on the hardware you're running it on cuz you could have a your 128 gig device could be an M3 Max or something like that or it could be an M5 Max. Do you know? So, the tokens per second is is completely dependent on the the hardware you're running it on. However, the interesting points to note here is dynamic one uh is 78% more accurate while being 62% smaller. Dynamic 2 uh 90% uh accurate while being 45% smaller. You know, quantization is this goal of running um or shrinking a model while maintaining as much of the intelligence as possible. There are many many way many ways to do that. Uh but yeah, the this is what they're running. So final thing to share is that when you go on hugging face, these are all you could you've got the original weights here. You've got GGUF version which is um a single package version which can run on max to be fair, but this is going to be massive. It's one 2.8 terabytes. Uh but you've got um for DJ DG Spark or DS Spark, whatever it's called. Uh here we go. That's what I was looking for. So, different quantizations, different trainings, people have tweaked it, people have done all sorts of nonsense through it. This is what open weights is all about. So, you can run it at home. Um, but you still need some hefty hefty gear. So, I was thinking about this the other day that like Kimmy K3 is is amazing. Like the the fact is uh if I show you again just to remind you like the fact that it's going toe to toe as I say with all of these models like it's next to Fable and GBT uh 6 5.6 all that sort of stuff. The fact that it's doing that it is incredible and these are like freeto use models. There's no clear uh path to monetization on something like Kimmy K3. So on the one hand, I'm like, you know, this this just shows that um basically the Chinese government are willing to fund this because they are the Chinese government are subsidizing the training and whatever of these open- source models that we're getting for sure, probably some um other investors and stuff like that. It takes a lot of money to train a model. So there's definitely a commitment there. However, this is a massive model and opensource is all about not necessarily all about, but it's all about flexibility and freedom and being able to run it locally and whatever. And I posit the idea in this little message here is if GLM or Quen or Gemma, one of the other kind of running models that you can run locally on your machine, if they were given infinite budget on like, yeah, make this as big as you like. I wonder if they a would be able to uh achieve the same sort of performance because this is this is doing really well but it's also massive. It's like what would be a good analogy? It's something to do with money probably but of course like if you plow more money into something it's going to get better. You're going to get more out of it. So, like I don't know. I I really like Kim K3 and I end it by saying I don't dislike Kim K3 and I like open source, but I wonder if the fact that it's so high up is distracting from the fact that actually we don't we lose a lot of the benefits of open source purely by it being so faking big. All that to be said, Kim K3 is now open sourced. uh you can go download it, you can do whatever you want with it, you can train it, and if you have good enough hardware, you could potentially run the one bit quantization version on your Mac Studio plus some other device. This is part of a larger conversation on my show, Command AI, which we stream live every single week. We discuss the news and all things related to AI in the world of design and web. Catch us next week and join in the banter. Command AI.