Mira Mirati's Open Source Model Is Not Very Smart, But That's Not The Point

Mira Murati, the CTO of Open AI, uh spread her wings and flew away and created a new startup called Thinking Machines released its first open-source model called Inkling. Command AI. It challenges China's dominance in open weight AI and signals a major new player in the open-source space. You might have You might have remembered um her from when they got when she got asked about Do you remember this when she got asked like where did they get the training data from? She was like she kind of pulled this face. No, I I didn't see I wish I would have. Yeah. We did We just stole it like everyone else. Yeah. Yeah. So, I haven't had a look too much into this. I saw I mean, it literally got released yesterday. And you know, 975 billion parameters under just under a trillion. You're not going to be running this necessarily locally. Um and in fact, let's see. Maybe this Let's hop on Hugging Face. Files info. Here we go. What's going to I think it's going to break it down. a little bit. Total size. Give me Give us that in bytes. How much is that in bytes? So, this is nearly 2 terabytes this uh this If if my calculations were done correctly, it's it's nearly 2 terabytes. So, it's I think it's possible to run to run locally um but a quantized version for sure. This is going to cost you a lot of money. What I tend to do if I go to Hugging Face, doubt anyone's done this. Oh, someone has quantized it. Amazing. So, someone's quantized it for 4-bit here. Okay. And this comes out at half a terabyte here. So, this this can run on two two Mac 512 Mac Studios. Which is pretty cool. So, you can run it. Yes, yeah, but you need something. Oh, yeah. This is This is Yeah, beefy. Many, many thousands of pounds. That's the That's the thing with local AI. Like, it's it's not something you can do on a budget. Like, people get excited about these small local models and and fair play. Like, I get excited about them. I love local models that can run on my, you know, pity pity 128 GB VRAM M5 Mac Studio. I love models that can run on that, but it's just Yeah. The The less you The smaller you go, the worse that model is. The less you can actually do with it. And coding models roughly start about about 30 30 um billion parameters. So, yeah. I mean, pretty cool. I've not used it. Can we get it on open um open router? Well, it's not on open router, so I don't know whether we can actually use it yet. On one benchmarks, the company says Inkling uses a third as many tokens as Nvidia's Nemo Tron 3 Ultra. So, it is it isn't like it's not the a massive model. It's not, as we'll get into, Kimmy K3 level of um compute needed for it. Um That raises the question of who within the enterprise market it's targeting. Thinking Machines is for now marketing Inkling less as for now marketing Inkling less as a finished product than as a starting point, something for organizations to fine-tune themselves through tinker. Interesting. This is part of a larger conversation on my show, Command AI, which we stream live every single week. We discuss the news and all things related to AI in the world of design and web. Catch us next week and join in the banter. Command AI.