Agents.md Explained: Smarter AI Coding Setup

some really interesting research that Versel has been doing on how skills or a simple agents MD file outperforms skills. So skills are again um a resource of best practices and how to use certain things. We know from open claw uh that it it basically enables it to use Google places or Twitter. you know, out the box, the LLM might not know how to use Twitter, right? It might not be trained on the newest updates on Twitter. Here's uh a great little explanation. AI code rely on training data that becomes outdated. NextJS introduces APIs like use cache connection forbidden that aren't in the current model of training data. So, Opus doesn't contain this information. When agents don't know these APIs, they generate incorrect code or fail fall back to older patterns. The reverse can also be true where you're running an older version of Nex.js and the model suggests newer APIs that don't exist in your project yet. We want to fix this by giving agents access to version match documentation so you're always up to date. This is kind of why we give it XMTPs and things like that to try and just help it maintain and be up to date cuz webdev is up, you know, it's it's changing all the time and we're relying on I think Opus is up to September last year Opus 4.5. So anyway, skills are open standard to enable this whereas uh agents.mmd or claude.md is a you know marked down file on the root of your project uh provides persistent context to coding agents. Whatever you put in agents md is available to the agent every turn. So it's similar to a again claw uh open claw where the sole MD is included in every single request. So they did some they did some tests and what they were finding is that skills weren't being triggered reliably and this is a an issue right 56% of eval cases the skill was never invoked the agent had access to the documentation but didn't use it adding the skill produced no improvement over the baseline so they guess this is without skill this is with this with skill the same rate you know same pass rate [sighs] then they moved things over to the agents.md um we tried adding explicit instructions to agent MD telling it the agent telling the agent to use the skill and I've got a story here the improved trigger rate to 95% and boosted the pass rate to 79%. So actually it's a kind of combination of the two and I talk about this cuz basically in my root claw MD file for my system I have whenever a new feature whenever a feature is being asked of you use the uh exert MCP to research that tool or technology or whatever because I was finding my MCPS weren't being called. I don't rely I've got a couple of skills but I don't rely heavily on skills. But you know putting two and two together you might as well put in your rules if you've got something you know especially in a code base you know you can have it clawed MDS can exist across your computer across all projects or you can have it just in a single project. If you're using Nex.js and you've got NexJS documentation or Nex.js JS skill um react uh whatever you might as well add it as a line in your agents MD file. So whilst this is a very um formal way of actually putting it together and showing statistics, this is kind of what I've been doing already is is just telling the agent to use skills, use MCPS. They actually found here what's this 100% as right as the hunch that paid off. What if we removed the decision entirely? Instead of hoping agents would invoke the skill, we could embed the docs directly in agent MD. Not the full documentation, just an index that tells the agent where to find the specific doc files that match your next.js version. So in fact, it looks like that they literally download the docs into the codebase and tell it and point it to it. And I think then it gets 100% rate. Nice. Which is really good. It's funny because there's been mutterings about this way of doing things for quite a while when talking about MCPs and this and that because it's much better at writing code. So yeah, I think this is probably part of the setup process. Again, thinking about it, put in your claw MD file, every technology that we add to this in this package JSON, go and get the documentation and make a folder for it. add it in the clawed MD file. You can get the agent to automatically do that with every project. So, you don't have to actually go off and get the documentation. That's what you could do because here's the thing with um broad code doesn't read agents.md, which is what they talk about in here. They talk about this agent MD file. What I do is I I've got a short on this. I create my when you slash in it on a claw code project, it creates a clawed MD file. What I've got in my computerwide claude MD file is create a um agents.mmd file, copy all the contents from claw md and then make a sim link, delete the claw md file and then sim link clawed md to the agents.md. So then everything is just done in that agents.md file. And I've asked it to do that whenever I initialize a project and it and it works. So you can have a lot of commands or setup that you prefer in that claude MD file. So pretty cool research that Versel were doing. Um, interestingly, wouldn't they were the ones who released skills? [laughter] Didn't they just release this tool about skills and now they've been like, "Ah, skills don't work that well." [laughter] [gasps] Anyway, if anything, they could probably add something to this. when you MPX a skill, it adds a line in the agents.mmd to to check the [music] skill. So yeah, pretty cool. Pretty cool. This is part of a larger conversation on my show, Command AI, which we stream live every single week. We discuss the news and all things related to AI in the world of design and web. Catch us next week and join in the banter.