What Makes a Good Skill.md?

What the research says agent skills need — and what it still gets wrong

Image generated by Gamma.ai

---

Four papers published this year measure whether agent skills actually work, and they all complicate the common advice about skills. It’s not necessarily how long the skill is. Here’s what works and how to spot what doesn’t using structured approaches and information types together.

---

I’ve spent this summer building a structured knowledge base — not just files I upload into AI, but one that helps me manage and organize them.

This has been the ghost I’ve been chasing, mostly through a graveyard of personal knowledge management apps. Every single one ended up in chaos.

This year I stopped looking and started building my own instead.

One working piece of that is the pipeline connecting my knowledge base to Blue, the project tracker I schedule this newsletter through. I published the skill behind it in Context Lab a few weeks ago. It’s called cyborgs-briefing.

This week, though, I was deep in the unglamorous part — revising those skills, reorganizing the documentation, and figuring out which had quietly gone stale.

What is a skill?

If you haven’t worked with these, it probably sounds more complicated than it actually is.

Basically, an agent skill is a plain markdown file, usually called SKILL.md. It sits in your folder and tells an AI agent how to do one specific job. You can also install them directly into AI tools like Claude and ChatGPT.

That makes skills documentation, with all the maintenance problems documentation has always had.

The format is simple and doesn’t really change across AI models.

A few lines of metadata at the top say what the skill is called and when it should be used. Below that are the instructions in natural language. A skill folder can also hold supporting files:

• scripts the agent can run,

• reference documents it can pull from,

• templates and examples.

But the SKILL.md file is the only required part.

What makes it different from a prompt is persistence and scope. A prompt is something you type once for one conversation. A skill is written down once and read every time the job comes up, by whatever agent happens to be doing it.

That makes it documentation, with all the maintenance problems documentation has always had.

Years ago I wrote about the anatomy of a prompt — how a good prompt separates task from context from content instead of mashing all three into one paragraph. A skill works the same but at a larger scale.

How my skills actually fit together

Here’s my drafting process, simplified:

1. A draft starts. Sometimes I write it from scratch, sometimes I ask AI to draft from my notes. Often it’s both.

2. I iterate on the draft. Often section by section or by task.

3. Skills run passes on it. One audits for AI-sounding language. One scans for missing links or citations. One polishes the opening and writes the AEO summary.

4. I revise by hand. This customizes the text. I also track my revision habits and use that to help AI improve its revision skill.

5. Publish. This is more than just posting on the web. I have AI incorporate meta-data and then organize it into my knowledge base.

But as I continue to fine tune how skills support this process, I’ve been looking for research on what actually makes a skill good.

The research has interesting findings, but there’s a blind spot.

What the research actually found

Li and colleagues’ SkillsBench is the most direct evidence available.

They tested 86 real tasks with skills mounted versus without. Average performance improved, but here’s what matters: agents cannot reliably write their own skills.

When models tried to generate skills for tasks they’d been given, performance dropped. The skills need to come from someone else — someone who understands the task deeply enough to document it.

Let’s call that person a writer.

More generally, the benchmark shows three patterns:

1. Skills can backfire. Some tasks got worse when a skill was attached. The skill prescribed a single heavyweight process with no way to opt out. It fired every time, even when it shouldn’t. A skill with no off switch can do more harm than no skill at all.

2. Focused beats comprehensive. Skills that tried to cover every edge case and scenario underperformed. Tightly scoped skills, with constraints clearly marked, outperformed ones that attempted to handle every variation.

3. The agent has to sort what it’s given. More guidance doesn’t mean better performance if the guidance is tangled. The agent has to separate rules from steps from examples before it can use any of them. That sorting is overhead.

Wu and Zhang frame skills as procedural knowledge: knowing-how, a different kind of content from knowing-what. And Jiang and colleagues formalize a skill as four parts:

• a condition for when it applies,

• a policy for what to do,

• a condition for when it’s done,

• and an interface so something else can call it.

Ling and colleagues took a different angle. They looked at what people actually publish: 40,285 skills in a public marketplace, heavily concentrated in software engineering. What they found was “widespread intent-level redundancy” — the same capability, rebuilt over and over under different names.

You need writers, not just models

In the end, agents can’t generate their own good skills. When a model tried to write a skill for itself, performance tanked. Skills need to come from a human who understands the task deeply enough to document it carefully.

The bottleneck isn’t the model—it’s you, the writer. And you need a method.

Every finding above is a symptom of the same underlying problem, and the papers keep describing it with the wrong instrument.

SkillsBench measured length. Not because length is the actual problem, but because that’s what you can measure automatically. But the real issue is five different kinds of content get stacked into one file.

That organization is work an actual human writer should have done.

I’ve argued before that a skill is documentation, not a prompt. Documentation has a methodology for this, built over decades. Content gets sorted by the job it does for the reader. This is how writers have always organized complex information.

I use mostly five types.

Table describing five information types: Concept, Reference, Principle, Process, and Task

When I restructured my own pipeline, I applied this grid to every skill.

The cyborgs-briefing skill that orients a drafting session is pure Concept and Task, which explains what a briefing is, then walks through the steps.

The structured-note skill I published this week sorts incoming content by type before it goes into the knowledge base — reference notes, concept notes, principle notes, and so on.

Both work because every block has one clear job. Both fail instantly if you mix the types without thinking about how they all connect.

Under this lens, all three failures reported in this research can be traced back to missing Principles.

Jiang and colleagues describe a skill as needing an entry condition (when to start), a process (what to do), and an exit condition (when you’re done). Missing that exit condition? The agent keeps working past “good enough.”

That’s a missing Principle. There’s no rule for when to stop.

The thirteen tasks that got worse? Same thing.

The skill had no boundary. There was no Principle saying “don’t use me if X.” So it fired every time, even when wrong.

And Ling’s marketplace redundancy happens when Principles aren’t separated from Process. Every small variation in the rules spawns a whole new skill file instead of an update to a shared one.

Manny Silva arrives at the same place from the practitioner side. I reviewed his book, Docs as Tests with AI, here in May. He argues that a skill needs three things — entry criteria, process steps, and an output specification.

He also argues that documentation should load in layers.

1. Metadata first — so the agent knows if this skill even applies.

2. Instructions next — what to actually do.

3. Resources last — details you only need if a step calls for them.

Separating these out into organized chunks helps both humans and readers understand and implement these skills.

His framework maps neatly to the five types.

• Entry criteria are Principle.

• Process steps are Task.

• Output specification splits into Reference (the shape) and Principle (the standard it must meet).

I would add to this anatomy a Concept type, making sure the AI understands key ideas about the task that are contextual and may be different from its training or what it finds in RAG or on the web.

Tacit knowledge is one of the important things the benchmarks can’t measure.

In his book, Silva describes a skill built from everything a practitioner consciously documented about her process … and it still failed.

She performed one comparison step so automatically she never thought to mention it. No amount of type-checking catches what you don’t know you know. That takes testing against real work.

Try it yourself

Try this the next time you are working with a skill. Go through your skill draft and label every block with the job it’s doing.

• explanation (Concept),

• lookup data (Reference),

• a rule (Principle),

• a flow (Process),

• or a step (Task).

Anything that doesn’t fit one of those five is either in the wrong place or shouldn’t exist. Anything that mixes two types should be split.

That’s it. No length audit needed. No word count. Just ask what job each block is doing.

Next: Inside a failing skill

Now that we have the framework, the real question is simple. What does a failed skill look like up close?

I pulled a real one from the SkillsBench benchmark that lost 13 percentage points of performance.

In the next piece, I’ll walk through that skill block by block, showing you exactly where it breaks. Understanding the problem is one thing. Seeing it in action is another.

I would love to hear what skills you are working with in the comments!

---

Sources

• Xiangyi Li et al., SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks (arXiv:2602.12670, 2026)

• Yanna Jiang et al., SoK: Agentic Skills — Beyond Tool Use in LLM Agents (arXiv:2602.20867, 2026)

• Yaxiong Wu and Yongyue Zhang, Agent Skills from the Perspective of Procedural Memory: A Survey (TechRxiv, 2026)

• George Ling, Shanshan Zhong, and Richard Huang, Agent Skills: A Data-Driven Analysis of Claude Skills (arXiv:2602.08004, 2026)

• Manny Silva, Docs as Tests with AI — my review

• citation-check task and citation-management skill, benchflow-ai/skillsbench (GitHub)