Human Beats AI in Coding Challenge

I'm not that surprised that the human programmer actually won. Human coder beats Open AI in a marathon programming contest. Polish programmer presly defeated OpenAI's advanced AI model in a 10-hour coding marathoder world tour. 1.8 8 trillion points uh compared to AI's 1.6 trillion. The atcoder world tour finals 2025 took place in Tokyo, Japan as a international on-site programming competition featuring the platform's top 12 performers from the 2024 ranking system. 10-hour heristic contest on July 16th where competitors tackled an NP hard optimization problem and a 5-hour algorithm contest. I'm not that surprised that the human programmer actually won. But it's still very interesting that in aspects like even this uh in a 10h hour where something like AI would you would assume that can win because it's 10 hours of coding. Yeah. And AI doesn't get tired. It's just you know more forest to be burned. The thing with LLMs these days LLMs are um really good at replicating what they learned. I mean they're literally just um statistical models which predict the next to where we simplify it and the thing generally is with um now this maze issue is the LLM may basically know okay I have these algorithms for pair finding I can apply this and this but it doesn't it doesn't know the optimal way to combine multiple algorithms together like chain them together or how to optimize for a specific problem with a specific range my assumption is that the LLM quickly had a solution working quickly iterated on But to to to get to the best solution, it would basically um need like much more time, many more let's say execution branches to play around. I don't know if they used here a single instance which did all the things if they had multiple instance running concurrently testing all possible solutions. But um basically in a smaller scale I expected to be honest open to win that they did not win is also understandable though. But I can I what I can tell you this though if this competition would not be in 10 horse but let's say a much more complex problem and would be over let's say a week or month the AI would have definitely lost. Yeah I no doubt so is this context that is so huge that we still can do this like small level but also very abstract and very huge computing. In my ideal scenario, I would imagine that if let's say an LLM executes let's say a program, it will compute after each line what the state of the program could be in a range of things similarly to how for example Typescript union types work. Yeah. And I could imagine if the LM for example use these use such a system and narrate it down it could actually get to an optimal solution. right now uh the way the models are they can test the code they they can just write it but they mostly like test and uh like they imitate it but they don't have an actual state with each line of code and I think that's one of the big issues we have right now especially let's say if you work in a large code base and you have an LLM it does it puts some random functions it puts some random data because the LLM actually doesn't evaluate the LSP the data it's just like it just makes stuff and just those assumptions and we need to AI generally needs to work more like we give you information and these are the information this is the truth and you can only work with this truth and that's like still lacking I think it's very interesting with um some models like they have tool calls so they can search the internet for example to look look up something and they say they do but they actually end up not doing it so Jim and I had that issue severely where basically um it says hey I should use this to but it doesn't look because it was not properly trained in but now we have new models like Kim V2 and Grock which have that properly trained in and I'm much more eager to call it um TheoG a different YouTuber as an example he also um ran uh two testing with Kimmy V2 and he did not run into a single issue on his benchmark so if that is true then Kimmy V2 is even more reliable with two calling than uh cloud cloud code and tool calling is literally the AO to work on agendic workflows and let's say let calling assistance everybody talks about tool calling What is tool calling? We give the AI a couple of doors. Each door has a description on what it does and the AI can decide to go through the door and trigger something. It's like um basically a list of functions or list of APIs and the AI can use these APIs in in his messages. You have an agent and you ask it about the weather and then in its tool called um list it has a weather tool and then you can just say it basically you define like you short description what this tool does. Let's say it gives you the current weather and then you describe what kind of information it needs like city and street. Then the AI could ask you, hey, I need it for this city and the street and then it's basically shows an API call and we get a response back and that response is then incorporated into the next message. So the AI has a first message where it says which where let's say things internally tells me which tool records it needs and then a second message it responds to the user. Yeah. Okay, that makes sense. This is part of a larger conversation on my show, Command AI, which we stream live every single week. We discuss the news and all things related to AI in the world of design and web. Catch us next week and join in the banter.