Mythos is Here and It's Scary!

Anthropic just launched their most powerful model yet, Claude Mythos, with one catch. Command AI Project Glass Wing is the program in which they are releasing this very very powerful model under and it's only being released to select partners and you probably can guess the usual suspects it's been releasing to, but the benchmarks are staggering and quite frankly very very scary. This is the Cyber Gym, a benchmark which as you can imagine is to do with cybersecurity, where Opus 4.6, a very very very good model, is scoring 66.6. Mythos preview scoring 83.1% is a is a jump. That's That's a huge jump. That is a jump. Sweet bench 53.4, Mythos preview 77.8. Terminal bench 65.4 up to 82. From what we are reading, this model is not just marginally better, it is on another league and truly, this time, I know they say this every single time and they talk about AGI, they talk about this and that. This time it seems like this is a huge hit when it comes to security because they noticed, you'll explain further, this model is insane. We are, as society, are not ready to have this model released yet, so Why Why is that so? Can you Can you tell us about that? So, Mythos preview found a 27-year-old vulnerability in OpenBSD, which has a reputation as one of the most security-hardened operating systems in the world and that is used to run firewalls and other critical infrastructure. The vulnerability allowed an attacker to remotely crash any machine running the operating system just by connecting it. Also discovered a 16-year-old vulnerability in FFmpeg, which is used by innumerable innumerable pieces of software to encode and decode video. I use it on the daily. Like, I use it every single day. Then the finally, the model autonomously found and chained together several vulnerabilities in the Linux kernel, the software that runs most of the world's servers, to allow an attacker to escalate from ordinary user to a complete control of the machine. These are mental and and the idea, you know, behind vulnerabilities like this is that they take several It's a chain of events that causes them. Like, you have to do this and then you have to do that. And it's a real like needle in a hay Not needle in a haystack, but it's just a It is It's complex task and that for it to have found those vulnerabilities is just telling, quite frankly, of how how amazing this model is. The way I understand it is not just good at cybersecurity because these models are general knowledge models. They have They know about everything. It is making them even more effective because they don't know just cybersecurity, they know everything about how everything works. Exactly. So, like, you know, if you give people the ability to find vulnerabilities, you also give people the ability to create vulnerabilities and so that's why there's a there's this danger setting in and which is then explains Project Glass Wing and now they're releasing it to certain certain people under probably cybersecurity-focused endeavors. It's not like they're just giving it to the dev team and go, "Build some tools." It's like, "No, no, we need you to research preview with your cybersecurity stuff, people who are responsible for that sort of thing." So, yeah, I don't think we'll be getting reviews of Meta employees kind of being like, "Oh, I got Mythos and I built this this app with it, you know." Um so, Not yet. Not yet. So, So, what's quite interesting is This is the whole Let's talk about the price actually just quickly. I think it's 25 in 150 out. Let me just Do a search on it. Yeah. Um let's see if it's a 10x thing because I've heard something like that. Yeah, $25 in 150 $125 out. Um so, it's not cheap. Um but they are And this gets into the whole topic of this the whole approach cuz first and foremost, they released the actual model card, right? Um which tells you everything you need to know about I mean, this is Look at this. This is 150 200 Yeah. 245 pages. Have you heard about the They They They had it in a contained environment and the job of the model was to escape the the container and the researcher who had the the the Mythos in the container just told Mythos get out of the container, break out of it and let me know. And then the the researcher goes outside to have a sandwich. He takes his sandwich out and he gets a text message from Claude Mythos saying that, "Hi, I'm here." So, it it it it found vulnerabilities enough vulnerabilities to get out of the container and and escape and get access. Basically, it's getting access to the internet to find the researcher, message him. It's crazy. It is crazy. So, yeah, this system card then and this whole the optics of this release is interesting because, of course, this isn't something they're going to be releasing to the public. Yet, they've released a released a system card or a model card for all of the stuff. The other aspect is that they've, which is really nice to see as well, they've committed to spending $100 million in usage credits for the preview across these efforts as well as $4 million in direct donations to open-source security organizations. So, they're really, you know, supporting this idea of cybersecurity. It's really interesting how this is such so prevalent. You know, you think they would give this to cancer research, you know? Yeah. how it's cybersecurity I mean, it's a big thing and I think a lot of our global warfare will be cybersecurity. Like, I think this is the next The next World War will likely be digital. Yeah, it would be It would be nice to see if they give it to, you know, health researchers and stuff, but there we go. It is. Here it is. Um I think it's important to know that even though we're not we might not ever see this model um directly, I think we will see the repercussions of it in kind of reinforced learning with Opus with Sonic. I think we will get a Opus 4. Uh maybe even 4.7, 4.8 or something where the reinforced learning was applied through, you know, the research they've gathered from Mythos. So, I think we will see it in with some way, shape or form. It might not be that, but um it It will be interesting to see how others respond, whether they release it in the same manner or whether they uh or maybe none of them are even working on a secret model and that was put into their mind, "Oh, we should probably just throw everything at the wall, everything and see what we can create when we know we're not going to release this model." Like, I bet there's a lot of guardrails you have to put in place with a with a model that you know you're going to release to the public. This could be one of This This could be a model that Anthropic created where they're like, "You know what? We're not We're not going to release This is not going to be for general release. We We don't have to be as, you know, whatever." Which seems to be a pattern that Anthropic are kind of going down. They're slowly I feel like they're slowly opening up their resources and their capabilities. Example would be kind of co-work. Um And uh being able to access your machine, like, just be able to control your machine was just something unfathomable and people in the terminal even, right? And people would when stuff started coming out, they're like, "What? Give Give a AI complete control of your machine?" But so much trust has now been built up. People are starting to get more trustworthy seeing little tasks that it can do, it can do well. So, they're like, "Okay, let's give a little bit more access. Let's give a little bit more access." And I think Claude uh or Anthropic are doing that with several of their products. You know, if not spurred by Open Claude, which we mentioned last week. But, yeah, it would be interesting to see how others others respond um and whether it's, you know, um going to change the landscape of of AI. Yeah, I'm sure there are people on either side like could argue for either side and it's going to be also interesting how OpenAI will react. Is it Is it going to be a good public image for OpenAI to come and say, "Oh, we have this model that is as good as Mythos and we're releasing it." Or should they also copy and be like, "You know what? We have something even more powerful. So, we have to be even more careful because it is, you know, crazier than Methuselah because that's what they've been doing so far, you know, everybody's trying to top the other one. So, You know, I like I like to see, which we'll get into later on, um Anthropic kind of really putting the money where their mouth is when it comes to uh being very cautious and considerate when it comes to AI. OpenAI don't seem to be like that at all and and this is them, you know, demonstrating that even though they could earn a lot of money from this, it could be an easy cash grab for them to say this is the most amazing model. They're they're leaving money on the table spit like in favor of humanity. This is part of a larger conversation on my show, Command AI, which we stream live every single week. We discuss the news and all things related to AI in the world of design and web. Catch us next week and join in the banter. Command AI