Is GLM-4.5 The Next BIG Thing? w/Max @warmwind_OS

This is absolutely definitely a really cool direction it's going into. The the hype around this model isn't quite there yet. The new Chinese open- source model GLM 4.5 was released this week and this is positioning itself as a high-end open source hybrid model uh targeting reasoning coding and agentic tasks and it's uh 355 billion parameters. It uses uses a mixture of experts approach. I don't know what that is. Go for it. Yeah. when you usually prompt an LLM it has basically like a huge let's say let's say a map and the map is like really big just to navigate it and mix is basically in the beginning we have like a router which depending on what comes in decides which of these maps to use and then for each generated tokens we just use a much smaller map so that's the map is like the active parameters and the total model size is like all the parameters at the time only one of these uh let's say chunks can be active and process the data and it loses a bit of capability ilities. But the huge advantage is that you have a very huge speed uh improvement because instead of having to evaluate always with all the tokens with all the parameters, you always work on with a subset which is special which is like the most likely to be correct for the kind of tokens you put in and you mod get trained. They roughly use all of the um experts equally. That's a very simplified explanation. Oh, okay. Fine. Yeah, makes sense. And you said you looked into this yesterday, right, Max? Currently what is the easiest and fastest way for us is to post train um open source models. So post- training means we basically ignore a lot of the training um take um take into account that it will probably lose lose a lot of knowledge but it can do what we want way better after um so we currently use Quen or Qen or whatever you want to call it. um um but not the news like not the three um but 2.5 uh VL because we want vision and we but what what we are currently exploring is maybe you can have some reasoning and planning without vision so that's some experience internally currently um but when we look at GLM 45 um it's unfortunately not vision the only GLM that is is GLM 4V that can input that can take um vision input And from from that perspective, it could be interesting for us to fine-tune it to get some reasoning done or test planning. But to to go go here and and say, "Oh, we are way better with SO3 or uh or or Claude Opus is kind of an um I mean it's it's it's fine the comparison, but it's like without vision. So if you if you don't have to support vision, you can basically throw out a lot of computation. you don't need to train for for for a huge new complex kind of task set. So in my compar I I I would say it's way easier if you like if you do it like them just throw away vision and then say okay we train it all on text. It's obviously easier to to outrun these multimodal models. In regard to also GM there's a general issue that the OGM4 and I run it locally. Uh the issue is basically that it has like huge instabilities of the context size. So you cannot even get past 8,000 tokens and stay stable. It already starts slowly break up at 6,000. What do you mean with break up? Basically it's it's getting the model gets very irrational and the recall like stick in the needle when everything if you look at the graphs it's horrible. It goes downhill really quickly. We generally have issues as an example. The new Gwen 3 was also overbaked where if you just train it with random noise data, it actually gets better again because it was just so much fine tuned like basically overbaked if if you like try to overoptimize it. It ranks really well. But this is on Chinese like in Chinese as a language cuz I was looking at the um LM Arena scores and I just couldn't find GLM 4.5. I didn't know where it was. And then I read somewhere or heard somewhere that it's it's it's these are Chinese benchmarks using Chinese language and things like that. So this is absolutely definitely a really cool direction it's going into. Um and something to get somewhat excited about, but I I think the the the hype around this model isn't quite there yet because it doesn't cater for Western language even. You know, this is part of a larger conversation on my show, Command AI, which we stream live every single week. We discuss the news and all things related to AI in the world of design and web. Catch us next week and join in the banter.