Read more

OpenWork is a free desktop app that reads and writes the markdown files on your own hard drive — the open-source counterpart to Claude Cowork. This walks through installing it, choosing a model without handing your documents to a training set, and turning a messy folder into topic domains you approved.

Most advice about getting AI to work with your own writing starts with a product. Upload to this, sync with that, connect the other thing.

They want to lock you into a product, so I’ve been going a different route … a local folder of markdown files that I can manage across AI tool sets.

Your files stay on your hard drive as plain text, in folders you named, and an assistant reads them off the disk the way you would.

You may have heard of this idea … building an LLM wiki as a second brain. But Karpathy’s version of this idea has the LLM build and maintain the wiki while he stays out of it.

I work the other way round. I want control of the knowledge. The assistant does the grunt work, but doesn’t decide the structure. Through agent documentation, I can control the shape of knowledge and keep myself in the looop.

What you need for any of this is a tool that can read and write files on your own machine.

Claude Cowork and ChatGPT Work both do it and both cost $20 a month, which is probably worth it.

OpenWork does it for free. This video walks through setting it up, and ends with a folder of my research PDFs sorted into topic domains I approved.

[video]

• 1:14 — What these files actually look like, in Obsidian

• 3:13 — The OpenWork interface and workspaces

• 4:43 — A real question against my teaching vault: write an AI policy for a workshop I’ve never run

• 6:32 — Download and install

• 7:18 — Choosing a model, and the privacy question

• 9:46 — Permissions, and pointing it at a folder

• 11:12 — When it doesn’t work: two things to check

• 12:21 — Turning a messy folder into topic domains

How do you choose a model when using OpenWork

OpenWork isn’t locked into a model. They have their own open source models, but you can also connect to other models.

If you already have an OpenAI or Anthropic key, use it. You pay what the API costs, and that provider’s privacy terms are the ones that apply to everything you do in OpenWork.

That’s the route I’d take if any of your material is sensitive.

The free models through OpenCode Zen are better than I expected. I used Big Pickle for everything in this video. Slower than Claude, but the quality holds up.

Here’s the catch. Your data may be used to improve the model during the trial period when using these free models. That includes training.

In this tutorial, you’ll point this at a folder of your own documents, so if any of it is client work, unpublished writing, or personal notes, don’t use the free tier.

Published material, or a folder you put together for the experiment, is fine.

Third option is Zen credits, $20 minimum with auto-reload at $5 and allow you to pick and choose from any model (including Open AI, Claude, and Gemini). So if you like playing around between models, this is a good choice.

Why can’t open work access my folder?

If you have trouble accessing folders with Open Work, its usually one of two things.

Your operating system has to grant OpenWork access to the disk, which happens in your privacy settings and looks different on every machine.

And if the folder lives in iCloud or another cloud service without being kept downloaded locally, the assistant sees file stubs rather than files.

Keep the folder on your hard drive.

What comes after setup?

The tutorial ends with a set of folders. What it doesn’t have yet is the part that makes retrieval reliable — the documents that tell your assistant what each folder is for, what to read first, and to show you a summary table before it touches anything.

The full activity sheet from the Kraków workshop — all six activities, every prompt written out — is available to paid subscribers here. It also lives in the Structured Skills Library alongside the other skills.

More to come in future posts!

If you build something with OpenWork, tell me in the comments!

Subscribe now

I experimented with a search of my own knowledge base before teaching a workshop on building one. It swept 999 files, opened 21, and missed every folder where the answer actually lived. Here is what changed on the second run, and the six-activity sequence I ran in Kraków to get there.

I spent this summer building something I could use to explore how structured content works with AI at scale ... while also providing my AI tools with a more nuanced knowledge base that is easy to access, but private and secure.

This meant building a markdown database on my hard drive.

If you don’t know what markdown files are, they are simply text files with simple notations that indicate formatting and content types.

In September I took it to the CAKE conference in Kraków and ran a ninety-minute workshop where people built one from scratch.

Before that, I wanted to test how AI actually works with this kind of knowledge base.

It’s been pretty clear that structure changes retrieval. How you organize files determines what an AI finds when it goes looking.

Easy to assert. Harder to show, because retrieval happens out of sight.

You see the draft, not the path to it. So I built a way to watch the path: a scan of my own vault that logs every thing AI does as it searches the database.

The test task was writing an AI policy for a new class ... something I write and revise every semester (and something my knowledge base ought to be good at).

Tracing how AI uses my knowledge

I have written those policies before, taught with them, and published about them.

The first run had the whole vault without any guiding documentation (no AGENTS.md, no domain map). It swept 999 files and opened 21. Not one of the 21 came from pedagogy or Teaching, which is where much of my information on this topic lives.

What it opened instead was adjacent material. Newsletter posts about AI and writing. A conference note. Documents that share vocabulary with an AI policy without being one.

The draft that came back was okay, but not quite right.

The second run had the same vault, the same model, the same prompt, with governing documents added:

• a file telling the assistant what to read first,

• a map of what each folder holds and who it’s for, and

• specific skills for how to use the knowlege.

It swept 349 files and opened 10. Six of them were actual AI policies.

The counts are the easy part to report. What matters is what came out.

Here is the policy the first run wrote, trimmed but not otherwise touched:

Writing has always happened in a network between people and technology, and AI is now part of that network. In this course we will treat AI as a tool that can augment your thinking rather than replace it.
← ai-and-writing/ (public posts)

Students bring different mindsets to these tools — instrumentalist, formalist, humanist, technologist, posthuman — and part of our work this semester is recognizing which one you hold.
← My Students Are Mostly Instrumentalists About AI.md

AI systems retrieve fragments rather than whole documents, so the structure of what you give them shapes what you get back. Consider what kind of information you are asking for — a concept, a reference, a process — and structure your prompt accordingly.
← AI System/Information Type Decision Guide.md

Read it as a student. It is about AI and writing. It tells you nothing you can act on: no permission, no limit, no line you could cross.

That third paragraph is the giveaway ... it’s retrieval methodology from my own system folder, dropped into a syllabus, where it answers a question no student asked.

Here is the second run:

AI can both hinder and enhance our capacity to learn. We need to be mindful of when it impedes us and when it gives us new understanding. This class is a safe space to experiment with AI without shame, fear, or guilt.
← A Heuristic for Writing Your AI Syllabus Language.md

You will not be penalized just for using AI in this course. Unless I say otherwise for a specific assignment, use AI writing assistants freely.
← My AI Policy Wide Open With Guardrails.md

Generated text — permitted, up to 150 words per instance. You are the head writer. AI assistance — permitted without limit: ideas, outlines, feedback, audience adaptation.
← Give Your Syllabus AI Toggle Switches.md

I reserve the right to ask you if or how you used AI — not to get you in trouble, but so we both understand how the tech is shaping your writing and my teaching.
← What Students Asked After I Rolled Out My AI Policy.md

Same question, same assistant, same machine. The second one is a policy. It has a number in it.

It also has the sentence I rewrote six months after the first version, because students kept asking what the disclosure was actually for.

The run found the revision instead of the original, which was impressive.

We spend a lot of time talking about hallucination, and for this kind of work I think it’s the wrong word. Nothing was invented, really.

The assistant inferred a policy from content that was never meant to answer the question. The answer was still a good possible answer, but not customized to my work in the best way.

Worst of all, the AI had no clue. So if I didn’t know what was in that knowledge base, I would totally miss it.

The trend right now is to give AI all you got. Connect your notes. Sync your drive. Upload the folder. And AI will magically deliver wisdom and good answers.

But I started building this way for the opposite reason.

I wanted control over what AI could reach, which is most of why I separate storage, structure, and access in the first place.

What the scan showed me is that control isn’t only about what AI can reach. Something in the folder has to tell it what it is looking at.

Subscribe now

The original LLM wiki … and half the story

In June, Andrej Karpathy posted about how he’s been spending his tokens lately — less on code, more on knowledge.

He points an LLM at a folder of raw sources and has it compile a wiki: markdown files, backlinks, articles for the concepts it finds.

He reads the result in Obsidian. He queries against it. He files the answers back in, so his research accumulates instead of disappearing into chat history.

Two things in that post stayed with me.

The first is what he didn’t need. He went in expecting to reach for retrieval infrastructure: vector databases, embeddings, the apparatus that has grown up around getting AI to use your documents.

He found that index files and short summaries of each document were enough, at least at the scale of a hundred articles. Markdown in folders, with a few files describing what’s in the other files.

That is the same finding I got from the second scan. I was measuring why a search went wrong. He was noticing that a simple setup went right. Both point at the index or guiding documentation doing the work.

The second is where we part company. Karpathy says of the wiki, “I rarely touch it directly.” The LLM writes it and maintains it; his hands stay on the raw sources and the questions.

For what he’s doing, that makes sense. He’s compiling research on topics he wants to understand, and the wiki is scaffolding for understanding them.

My knowledge base is my teaching, my published writing, the arguments I’ve been making for years and the notes underneath them.

If an assistant reorganizes it according to categories it derived, I’ve handed over my expertise.

The domains in my vault are not a filing convenience. They’re how I understand my own work.

So I keep the taxonomy, but let the llm do the grunt work. The assistant converts, files, tags, checks, logs, and drafts. It doesn’t decide what the categories are, what counts as a principle rather than a process, or which folder a note belongs in when the answer is unclear.

Those are the decisions that make the knowledge base mine, and they’re also the decisions that determine what gets found later.

I’m still digging into this approach, but if you want to join the exploration, below are the instructions from my workshop that help you start from zero.

Paid subscribers get access to my live public skills library, along with all my developing materials on building structured knowledge bases.

Need something more foundational? Check out my course on Writing with Machines. Still free for paid subscribers, because I haven’t gotten around to changing that!

Read more

Documentation drift happens when the same piece of information lives in several places and the copies stop matching, and AI now exposes it by blending every version into one answer, choosing the wrong one, or just plain hallucinating. This first Q&A Kitchen uses a restaurant kitchen and a 100-document manufacturing case study to show three provenance checks for catching drift and where to stop it.

This issue of Q&A Kitchen is sponsored by Precision Content, which helps enterprise teams structure their content for people and AI.

Picture two cooks at two stations, both mixing the house spice blend.

One is working from the current recipe card. The other is working from last month’s, from before the chef swapped smoked paprika for chipotle and doubled the cumin.

Both blends go out, and each one tastes fine on its own. But a regular who orders the same dish every Friday knows something’s different the moment it hits the table.

Now the kitchen has a question it can’t answer. Which blend is right, and who changed what?

Documentation works the same way.

A product spec lives in a help article, a training deck, and a sales one-pager.

Someone updates one of them, and the other two keep serving last month’s recipe, even though each page reads fine on its own.

That’s documentation drift.

For years, the only people who noticed were the regulars ... a customer comparing two pages, a new hire getting conflicting answers.

Now there’s a new regular at the table. An AI system pulls from every page at once, blends them into one answer, and has no way to tell which recipe is current.

As Precision Content puts it, “AI does not fix content problems. It exposes them.”

I’m Lance Cummings, and welcome to Q&A Kitchen, a new monthly format here at Cyborgs Writing. Each issue takes one question content professionals are asking and pulls together what I’ve been learning, testing, and writing about it into a single answer.

Every section answers one question, so you can jump to the one you need. And if you’d rather watch than read, there’s a video version too.

And now that AI can scale your small business, or just your own personal knowledge base, documentation drift is an important concept for everyone!

So let’s start with this month’s question. How do you stop drift before AI finds it for you?

Subscribe now

Why does documentation drift?

Documentation drifts because the same information has to live in more than one place.

That part is normal.

A process model tells a manager what should happen. A checklist tells a quality inspector what to check right now. A training video walks a new hire through it.

Rhetoric scholars Clay Spinuzzi and Mark Zachry call this a “genre ecology,” or different documents doing different jobs around the same work.

The trouble starts when nothing connects them.

In 2016, researchers spent eight weeks inside the procurement department of a Dutch food manufacturer, one of the five largest in its industry, with more than 20,000 employees across 25 countries.

They traced a single process, approving a purchase, and found roughly 100 documents describing it:

• 30+ written work instructions

• 30+ slide decks

• 15 videos

• 20+ hybrid documents

• a handful of process models, spreadsheets, and e-learning modules

I’ll admit, the number that stuck with me wasn’t the hundred documents. It was the 120 obsolete ones a recent cleanup had already thrown away.

Meanwhile, one employee used an outdated specification to create purchase orders. The orders went out with the wrong supplier reference numbers, and the company paid for it.

That study is nearly a decade old and predates generative AI entirely.

But that’s what makes it useful. The authors traced the drift to three causes:

• Different audiences needed different levels of detail.

• Different purposes needed different formats.

• Nobody had a standard for capturing the information.

None of those causes came from technology, and AI hasn’t removed any of them.

Precision Content sees the same pattern in its work with enterprise teams. Drift tends to start right after new tools and training roll out.

A new tool doesn’t prevent drift ... it often starts the clock.

How do you know when content has drifted?

When the regular sends the plate back, the chef retraces the blend before she blames anyone.

First she checks the shelf. Is there one container of house blend, or are there three jars, each mixed from memory at a different station?

Then she checks the recipe. Which card did the cook use? Is it the current one, or last month’s still taped to the wall?

Then she checks the prep log. Who mixed this batch, when, and from which card?

Content professionals call this provenance, and the chef’s three checks are the same three questions:

• Is this fact stored once?

• Can you trace it back to what defines it?

• Can you prove you followed your own process over time?

Now go back to the food manufacturer.

The employee who sent out the wrong supplier numbers was working from one of roughly 100 documents.

Was that number stored only once? Could anyone trace that document back to the approved specification? Was there a record of who last changed it?

The answer to all three was almost certainly no. The organization only found the problem after the orders went out, the same way the kitchen only found the drifted blend after the regular tasted it.

Most teams know the first question as a “single source of truth.” It often gets read as one giant system for everything.

Mark Baker’s Structured Writing describes something narrower, one source per fact. Pricing can live in one system and specs in another, as long as the same fact never lives in two places. That discipline is the foundation Precision Content builds before any AI goes on top.

Every inconsistency turns your reader into an auditor ... and most readers won’t do the audit. They’ll just stop trusting the page.

Where do you stop drift?

A kitchen stops drift at the stations, where ingredients change hands.

Content works the same way. Facts move from a subject matter expert to a writer, from a writer to a reviewer, from a reviewer to a translator, and now from all of them to an AI system.

Most drift starts at one of those handoffs, when someone copies a fact instead of pointing to it.

If you are working as a small business or developing a personal knowledge base, then draft happens when you hand off work to your AI agents.

That’s the content supply chain. It’s the path a piece of information travels from the source to the reader or user.

Not every ingredient spoils at the same rate, though. Precision Content teaches teams to sort information by type to help triage content.

Their tax-filing example shows why that matters here:

• A concept like “a tax deduction is an amount subtracted from your income before tax is calculated” almost never changes.

• A reference fact like “the agency reviews filings within five business days” can change next quarter.

The concept is dry rice. The reference fact is fresh herbs.

You don’t need to watch both with the same intensity, but you do need to know which one you’re holding.

I sort my own teaching materials this way. Definitions and course policies live once in a shared folder and rarely change. Due dates and schedules are the fresh herbs, so those are the ones I check every week.

That’s where provenance comes in. Store the volatile facts once, trace them to whoever owns them, and log every change, because that’s the information the new regular at the table will pull first.

An AI system doesn’t taste the difference between last month’s blend and this month’s. It just serves both … or makes something up.

What can you check this week?

Pick one piece of information your team publishes in more than one place. A price, a deadline, a supported version number.

If you are a small business or working with personal knowledge base, think of something you use often, but in different places.

Then run the chef’s three checks on it. Is it stored once? Can you trace every copy back to whoever owns it? Is there a record of the last time it changed?

If any answer is no, you’ve found where your next drift will start ... before a customer, or an AI system, finds it for you.

Next month, Q&A Kitchen takes that third question further. How do you prove where content came from?

Got a question from your own content kitchen? Leave it in the comments or hit reply.

I’m Lance Cummings. Until next time ... watch those recipe cards.

Leave a comment

About Precision Content

Precision Content helps enterprise organizations turn scattered documentation into structured, reusable content that people can trust and AI systems can retrieve accurately. Their methodology, standards training, and content transformation services give teams a single source for every fact.

At CAKE conference 2026, AI agents showed up as a measurable second audience for technical documentation, while human readers grew too. Content engineering means automating the predictable writing, keeping the reporting and editing for humans, and teaching the judgment Aristotle called phronesis.

In February, AI agents made up 3% of the traffic to Box’s developer docs.

By July, they made up 58%.

Ian Freeman shared that number at CAKE conf in Kraków this month. Those agents read documentation so they can turn it into working code.

The next day, Anastasiya Biaroza showed her own analytics. Nearly half the readers of her developer docs are now agents.

Freeman also pointed out that human traffic to Box’s docs went up over the same months. The people are still reading, but a second audience has arrived alongside them.

Machines aren’t replacing writers. Content for humans isn’t going away. The audience just got bigger.

I’ve made some version of this prediction at conferences for four years. Writing was turning into something else, and I never quite knew what to call it. By 2024, the name I kept coming back to was content engineering.

Each year, I could point to job titles and hunches, but this year, I see more and more people using the term content engineering to describe what they are doing in the workplace.

Mainly ... managing the back end of AI writing systems and validating outputs. Tech writing is becoming less and less about “generating” text directly to users.

(Is it a coincidence? I’m getting a slew of tech writing job posts in my email this week ... and just saw one from Anthropic for a Content Engineer.)

This is one of my main takeaways from the world’s best content conference ... CAKE.

Hello, content engineering

The prediction started in Kraków, too ... with CAKE’s predecessor, Soap!.

At Soap! in 2022, I argued that new technologies would kill off writers but not creators. The people who last would be the ones who shape information.

It was a bit tongue in cheek, because I call those writers, too.

A year later, back at Soap!, I stopped calling what I teach “writing” and started calling it content and information development.

AI could already generate text, but deciding what a situation needed and how to put the content together was still a human job.

At ConVEx in 2024 (the best content conference in the US), I still hadn’t settled on a name: “I don’t know what to call it ... but it’s not tech writing.”

That was also the year “content engineer” started showing up in job titles more and more.

The work itself is much older than the title, though. Robert Glushko was doing document engineering in the early 1980s. By 2018, Mike Atherton and Carrie Hane’s Designing Connected Content had split the field into content engineering and content operations, and Joe Gollner had moved from calling himself a content strategist to a content engineer.

Rahel Anne Bailie’s uneven history of content strategy traces the whole lineage ... and I’ll come back to it in a future post.

By ConVEx 2025, the conversation had moved to purpose. I came home convinced that AI can imitate a goal but can’t supply one, and supplying it falls to the people who design the content.

That fall at CAKE, the argument turned to structure. The same semantic markup that makes content accessible to a screen reader makes it processable by a machine.

Every one of those posts described a shift I could feel but couldn’t quite measure.

Now I’m starting to see some numbers.

Drafts are cheap now

The first thing to go is the writing you could have predicted.

Freeman’s team figured this out quickly. Reference docs now come straight from the API specs, and tutorials complete themselves. That frees the writers to build structure, and for making the docs actually move developers toward signing up.

But cheap drafts create a new problem ... there are a lot more of them.

According to Justin Gagne, AI writing skips two roles, the reporter who finds things out and the editor who decides what’s worth saying. If you hand everyone a writer, the drafts multiply, but the reporting and the editing don’t happen on their own.

That’s where the human work has gone, and most practitioners already know it.

Małgorzata Lis surveyed 61 technical writers and IT professionals for her thesis and presentation, and they told her the same thing. AI is useful, but they don’t trust anything it produces until a person has checked it.

So here’s my working definition of content engineering. Content engineers:

• Design the process that makes the drafts

• Find out what the drafts need to say

• Check what comes out

It’s still writing work, but it relies on a paradigm change that sees content and information (or writing) as a system of people, processes, and technology.

This is a paradigm that has been around for decades, but is becoming a cornerstone of the tech writing field.

The machines want what editors always wanted

So, yeah ... agents are going to read your docs. The fundamentals, though, aren’t changing. They just become more important (and less forgiving when things aren’t quite right).

In her presentation, Biaroza’s analytics showed three things agents reward:

• A real claim up front

• Structure that means something

• Nothing padded

Here’s an example.

A page that opens with “Welcome to our file management guide” makes an agent wade through a paragraph before it finds anything usable.

A page that opens with “Upload a file with a POST request to /files” gives it something to act on in the first line.

Headings that name tasks (”Upload a file”) help. Headings that name themes (”Getting started”) don’t.

And the intro paragraph that exists only so the page doesn’t look thin is just noise to a machine.

Any editor would have marked up those same pages twenty years ago.

A human reader forgives a padded intro and scrolls past it, but an agent has to process every word of it, and each word takes up space in the context window and costs inference.

So the skill that matters is knowing, page by page, which standard applies where. Presenters at CAKE mostly called this judgment.

But there’s an older word from rhetoric that I keep coming back to.

Aristotle called it phronesis, the practical wisdom to make a sound call in a particular situation where no rule strictly applies. Today we’d call those situations probabilistic.

I saw it all over CAKE.

Freeman’s team had to decide which docs were safe to generate and which needed a person. Lis’s respondents check AI output because they can’t yet say in advance when it will be wrong.

Agentic workflows run on that kind of judgment, and so far only people supply it.

Can you teach judgment?

So ... if AI takes over the predictable writing, where does the next generation learn to make these judgment calls?

This is becoming the core problem in technical writing ... and it’s only going to get more important.

Junior writers used to build judgment on exactly the work that’s being automated. They wrote the reference page, got it marked up, and wrote the next one better. Take away the reference page and you take away the practice.

I’d rather not protect the busywork. I’d rather have people practice the judgment directly.

That’s what I tried in the workshop I ran at CAKE. Before anyone built a single note, I asked the room whether they had a taxonomy for their knowledge written down anywhere.

One hand went up.

So that’s where we started.

Everyone pointed their AI assistant at a folder of their own documents and asked what domains it saw.

The real work was figuring out which of those domains were actually useful for the person’s knowledge project.

The taxonomies that came back didn’t look like mine. This is a probabilistic task ... a rhetorical one. The answer will always be contextual.

One person organized by product feature, and others by user characteristics. None of them was wrong. Each one answered three questions only that person could answer:

• Who reads this folder?

• How do they find things in it?

• How fast does it change?

An AI can propose folders all day. Deciding which ones match how you actually think about your material is phronesis at a small scale, and you get better at it by doing it.

The agents will keep reading, and so will the people. Both need someone who has practiced making the important calls.

Soon, I’ll walk through the full workshop step by step for paid subscribers, from the first folder to the taxonomy to your first AI-ready note, so you can build one yourself.

I deleted most of the automations in my knowledge base this week, unable to tell which skill had run or whether two had stepped on each other. What survived passed one test: a clear goal, tight parameters, and a job I actually hate doing by hand. That’s what became the three-skill pipeline I now run to finish every Substack draft.

I deleted three skills this week.

Honestly, I kept tripping over them. One might contradict another, or one might work against what I’m doing manually.

For skills to really be useful, you have to be able to track your them, understand how they work together, and verify they are working as intended.

Sometimes a trigger might fire when I don’t want it to, or I forget to pull the trigger myself.

Each individual skill looks good on its own, but the more skills you have, the more work that is ... even if you have an agent keeping an eye on things.

I’d been building toward more automation for months in my local markdown knowledge base, on the assumption that more coverage meant a better system.

But as things grow, the system becomes less trustworthy, especially as solo “manager.”

I’d started double-checking outputs I used to take for granted, because I could no longer be sure which skill had actually run, or whether two of them had stepped on each other without my noticing.

So this week I went the other direction. I opened the folder and started asking, one skill at a time, whether I’d actually miss it. The answer was no for many.

What’s left is smaller and works better.

Subscribe now

What actually needs governing

Skills don’t fail the way scripts do. A script breaks and you get an error log. A skill misfires without any error, or doesn’t fire at all, and you don’t find out until later (if you find out at all.)

This is really what the Hugging Face incident was about, but at scale. The developers lost control of the agents, because they weren’t or couldn’t watch them.

I have a pipeline of skills that keep my knowledge base and Blue, the project tracker I schedule this newsletter through, in sync. In practice, I kept forgetting to trigger them, so the two drifted.

Then when I finally ran the sync, it had a backlog to work through and the output got confusing — too much happening in one pass.

Cards also kept showing up in Blue with “LinkedIn” tacked onto the title, no matter how many times I edited the skill that was supposed to control titling.

At first that made sense — an old version of the skill was still in play, taking precedence over what I’d just rewritten. So I cleaned that up. It kept happening anyway.

That’s when I realized the actual problem wasn’t a bug in one skill. It was that too many things had a say in the same decision — a skill, a folder-level AGENT doc, and whatever the AI was inferring from the existing structure of the workspace itself.

(An AGENT doc is usually a file named AGENTS.md or similar — standing context an AI agent reads automatically whenever it works in a given folder or project, separate from any skill you invoke by name.)

Three sources of truth, occasionally agreeing, occasionally not, and no way to tell from the outside which one had won on a given run.

The fix wasn’t a better skill. It was moving the title preference out of a skill entirely and into the AGENT doc that already loads whenever the AI works in that folder — because the AI could already deduce most of how to use Blue just from the structure I’d built into it.

A skill should be for a job that needs real instructions. A stable preference like a title format doesn’t need a skill watching over it forever; it needs to live where the AI already looks, once.

This requires being judicious about what becomes a skill, what becomes an AGENT doc, and what the structure of the workspace can already carry on its own without either one.

Judging a skill by what it does, not how much it covers

Once I’d cleared out the folder, the skills left standing were the ones doing a job with a clear goal, rigid parameters, and something I actually hated doing by hand.

That third piece is the one that’s probably my most valuable. Adding a skill for something I don’t mind doing creates more opportunity for drift, whereas a skill that automates something I never do, because it’s tedious or I keep forgetting it, is definitely worth the maintenance cost.

Clear goals and rigid parameters do the rest of the work.

A skill with an unclear goal starts making judgment calls nobody asked it to make — the same failure mode as the AGENT doc problem, just inside a single file instead of across three of them.

A skill with loose parameters runs when you didn’t want it to, on inputs you didn’t intend.

So I started asking three key questions:

• Do I know exactly what this is supposed to produce?

• Are the conditions for running it specific enough that it won’t fire on the wrong job?

• And is this a task I’d pay real friction to avoid doing myself?

If the answer to any of those is no, the skill isn’t ready — or shouldn’t exist at all.

The case study I actually built

There are a lot of things one does when you finish a draft for publication, even for more informal spaces like Substack. It’s also probably a collection of my least favorite tasks.

So these skills I’m keeping ... and I intentionally keep them as separate skills, not just one big one.

finalize-substack-draft runs a link pass.

It searches my own back-catalog for essays and LinkedIn posts worth backlinking, checks a running reference file for affiliate and referral links that fit, and flags named tools, books, and sources worth linking to.

substack-summary-seo does the opposite kind of job. It writes directly into the file: a visible summary at the top, in italics between horizontal rules, and an SEO description into the frontmatter.

And finally, gamma-featured-image-prompt writes one paste-ready prompt for the post’s feed image, tied to the actual title or hook, in the newsletter’s flat-line brand style, legible at thumbnail size. It doesn’t generate the image. I still run the prompt in Gamma myself and pick the result (affiliate link, appreciate it if you use it).

It might seem counter-intuitive. I just created or kept three skills instead of one. But if a skill has clear parameters, it does not add to the bloat.

This still relies on me to trigger these. Since the tasks are clear, it’s not as hard to remember, but I also created an orchestrator skill to manage clusters of specific skills like this.

For example, my revise skill runs three skills in order. Before it does anything, it analyzes for specific rhetorical patterns that I like to revise and only triggers the revision skill if that analysis passes a certain threshold.

So I decided to also create an orchestrator for the “finishing” tasks, in the same information-typed format I use for everything in the public Structured Skills Library:

---
name: finishing-skill
---

<!-- CONCEPT -->

## Concept

The finishing pass on a Substack draft isn't one job, it's three: find link opportunities, write the summary and SEO metadata, and generate a featured-image prompt. Each produces a different kind of output — a report to decide on, a finished artifact written straight to the file, and a prompt to run in a different tool entirely — so they stay three skills instead of one. This skill is the thin connector that runs them in order and hands back one combined set of results. It holds no linking logic, no SEO-writing rules, and no image-prompt conventions of its own — each of those lives in its own skill, loaded only when that step actually runs.

<!-- REFERENCE -->

## Reference

The three stages, in order:

| Order | Skill | Input | Output |
|---|---|---|---|
| 1 | finalize-substack-draft | the draft | a report of link candidates — backlinks, affiliate links, named sources |
| 2 | substack-summary-seo | the draft | writes a visible summary and frontmatter SEO description directly into the file |
| 3 | gamma-featured-image-prompt | the post's title/hook | one paste-ready Gamma prompt for the feed image |

<!-- PRINCIPLE -->

## Principle

- Run in this order and no other. Link candidates and the SEO summary both benefit from a draft that's otherwise settled; the image prompt only needs the title or hook, so it can run last without waiting on anything upstream.
- Never auto-apply a link candidate. Stage 1 produces a list; a person decides what actually gets added. Only stage 2 is allowed to write into the draft directly, because it isn't making an editorial judgment call — it's generating metadata from what's already there.
- Don't merge these into one instruction set. Each stage has its own reference file (a link taxonomy, an SEO style guide, a brand palette) that only needs to load when that stage runs. Combining them means every run pays the context cost of all three.
- This skill doesn't judge whether the draft is ready to publish. That call stays outside it, same as the rest of the pipeline.

<!-- PROCESS -->

## Process

This produces a bundle of pass-along outputs, not a single artifact — a link report, a modified file, and a prompt to paste elsewhere. Nothing here decides publish-readiness or applies edits beyond what stage 2 already does on its own.

<!-- TASK -->

## Task

1. Run finalize-substack-draft on the draft. Keep the report of link candidates — do not add any of them yet.
2. Run substack-summary-seo on the same draft. It writes the visible summary and SEO description directly into the file.
3. Run gamma-featured-image-prompt using the post's title or hook. Keep the returned prompt as-is; do not run it against an image generator.
4. Hand back all three outputs together: the link candidates (for a human decision), confirmation that the summary/SEO was written, and the image prompt (for the writer to run in Gamma).

The skills this one calls live in the public library too, for paid subscribers who want to see how they’re actually written.

Subscribe now

Building a system that lasts

This is why I don’t automate many things, and when I do, the workflow is clear and detailed with a clear goal and tight parameters ... and it has to be something truly valuable (and honestly, worth the risk).

I don’t want to scale simply by adding more. I want to scale by being willing to take things away, and by being honest about which piece of a system actually needs a skill watching over it and which piece just needs a place to live.

That’s what it means to be intentional about AI ... and it takes a lot of work. More work than many would like to admit, I think.

Anthropic’s own PowerPoint and Excel skills fail a real SkillsBench task, updating one currency rate in a spreadsheet embedded inside a slide, because neither skill carries the information types that would have caught it. I ran the benchmark’s actual grading test, reproduced the failure, and found the fix in one added Process and one promoted Concept.

Honestly, I’m still working out my own best practices around skilling AI agents ... it’s quite amazing how temperamental they can be ... and why I’m a bit nervous about “24/7 autonomous agents.”

And why I’m not that surprised by the Hugging Face incident. The higher the model, the harder it works ... and the harder it can go off the guardrails without good documentatation,

Right now information types keep me focused and organized, but I’m always testing that framework against what research I can find.

That said, it took me a couple weeks to find a skill worth analyzing that wasn’t too niche or complicated and provides a useful analysis measured against the SkillsBench research.

You still might have to bear with me, because this is an obscure PowerPoint task, but one that illustrates what the researchers called a “procedural gap” problem.

This involves two skills built by Anthropic that help with PowerPoint visuals and formulas.

Think for a moment about using a PowerPoint slide built for financial reporting with a currency exchange table sitting inside it that is not a native PowerPoint table but an Excel workbook, embedded with formulas and all. Next to it, a text box gives one updated rate. Change that one number, and other cells are supposed to update themselves.

Making that update by hand is tedious, and nothing does it automatically once a script is involved, so there are two skills an agent can use to solve this problem, one directed at PowerPoint and one at Excel.

A close analysis shows well-written and organized skills. The PowerPoint skill knows exactly how to unpack a file, edit its XML, put it back together. The Excel skill knows exactly how fragile a spreadsheet gets with these scripts and lays out the fix.

But neither file ever mentions the other. And office and white-collar tasks, the category this one belongs to, saw one of the smallest gains from skills of any category in the study -- 40.5% to 53.0% with skills added.

So I thought this might be a good one to do a deeper analysis on.

Subscribe now

Updating one number

Okay ... I can feel it. You maybe are still a bit lost on these skills.

“This skill does what?”

Let me give you a little more detail.

Let’s say you are presenting a company’s finance, and one of your PowerPoint slides hold a small currency conversion table with four currencies and their relative worth.

Half of that grid is typed in by hand. The other half is calculated so nobody has to keep two numbers in sync by hand.

The whole table is a working Excel file, embedded inside the PowerPoint file the way you’d paste a live spreadsheet into a slide so it stays editable later.

So why isn’t updating one rate as simple as it sounds?

First, a formula and its answer are stored as two separate things inside the file: the formula itself, and the last number it calculated.

Second, some of the everyday tools used to script a change to an Excel file, clear out every stored answer in the file the moment they save.

The script doesn’t do recalculations.

Every cell that depended on a formula goes blank if you skip a step.

The little picture of the table that shows up on the slide itself isn’t a live view of the spreadsheet. It’s a snapshot, taken once and stored separately, and it only gets redrawn when a person opens that embedded table directly inside PowerPoint and lets the program regenerate the image.

So “just update a number” turns into several separate steps, each easy to skip without anything obviously breaking: change the one cell, recalculate the whole workbook so the dependent cells catch up, and know that the picture on the slide might still need a human to open it once before anyone sees the fix.

Anatomy of two almost-complete skills

The paper calls this a procedural gap, or a place where an agent has the model capacity to do the work but not the concrete steps to do it correctly. So let’s read each skill for what kind of information is missing.

Two things stand out once you go looking for information types in these files — Concept, Reference, Principle, Process, Task, the five laid out in Part 1 of this series — instead of just skimming for policy.

Start with Excel. Its Principle is stated once, plainly:

“Excel files created or modified by openpyxl contain formulas as strings but not calculated values. Use the provided recalc.py script to recalculate formulas.”

The first half is a fact about how Excel files work. The second half is an instruction. They’re fused into one sentence, and only the instruction gets developed further, as a mandatory step in the file’s numbered workflow:

5. Recalculate formulas (MANDATORY IF USING FORMULAS): Use the recalc.py script

The fact about Excel never becomes a full on Concept or Reference that you can easily remember.

The PowerPoint skill has the opposite problem. Its Concept is right there in the first paragraph:

“A .pptx file is essentially a ZIP archive containing XML files and other resources that you can read or edit.”

Pretty clear concept. “Other resources” already covers an embedded Excel workbook.

But nothing built on top of that Concept ever asks what an “other resource” might turn out to be. The skill’s own Task for editing a file is four steps:

1. Unpack the presentation

2. Edit the XML files (primarily ppt/slides/slide{N}.xml and related files)

3. Validate immediately after each edit

4. Pack the final presentation

Both files are missing the same type: Process.

Both have plenty of individual steps already; neither has the overview that would sit above those steps and explain the context for the tasks and how they connect with a broader workflow.

There are Task headers labeled “Workflow” in both places, but those sections are tasks not workflows (or processes). Nothing above either of them ever zooms out to the whole process.

The Excel skill fails to provide clear definitions of the most important concepts in the skill, while the PowerPoint skill fails to contextualize the tasks with a process.

On top of that, neither one knows about the other.

Running the test

I didn’t want to take the paper’s case study on faith ... so I had AI run the task itself. Then I checked its work against the benchmark’s own grading test.

The real test file is public, so I downloaded the slide with the same kind of currency table sitting inside it that I described above.

I asked the AI to open the embedded spreadsheet, update the one rate cell, save. Then the AI ran the benchmark’s actual grading script against its own result. That script checks eight separate things about the finished file.

Six of eight passed. The two failures were about the inverse rate and the other, unrelated cells, which is the blank-cell cascade the skill is trying to avoid.

The tool the AI used to edit the spreadsheet erased every calculated value in the file once saved.

So I added one instruction.

Recalculate the whole spreadsheet before saving it, the way the skill’s own separate script is built to do, and only then close it back up inside the slide.

Same eight-part test, eight passes this time.

There was more to check, though.

I also asked the AI to check the actual picture of the table that shows up on the slide, the snapshot image I mentioned earlier. Comparing that image across three versions of the file, the original, the failing attempt, and the fixed one. It turned out to be pixel-for-pixel identical in all three.

Even the benchmark’s own official answer key, written by a human expert to define a perfect score, never touches that image either. A file can pass every single check the benchmark runs and still show the old exchange rate on the slide, unchanged, to anyone looking at it, until someone opens that embedded table by hand in PowerPoint and lets it redraw itself.

This is the kind of document drift we often seen in content management. More than one artifact references the same detail, but the “single source of truth” is under-managed.

The fix, retyped

Here’s what would close this gap in both skills.

Add one Process to the pptx skill, placed above its existing Task steps rather than buried inside them:

Process
Unpacking a .pptx file surfaces more than slide XML. Before editing anything found inside, check what kind of file it actually is. Slide markup gets edited directly, following the steps below. Anything else, like an embedded spreadsheet, an image, or a video, is a file of its own kind, governed by its own rules, and needs whatever skill handles that kind of file before you touch it or repack it.

Three sentences. It doesn’t change the pptx skill’s Concept, which was already correct, and it doesn’t touch the Excel skill’s Principle about recalculating formulas, which was already correct too.

It just gives the agent a reason to ask ... what is this thing I just unpacked?

The Excel skill needs a small upgrade, too. Pull the fact already sitting inside its Principle out into its own line:

Concept
Formulas and their last-calculated values are two different things inside an Excel file. Editing tools that rewrite the file, including openpyxl, can update one without preserving the other, whether the file is being edited directly or is sitting inside something else, like a slide.

I tested this pair of edits informally.

When given only the two revised skills and the same task, a second agent found the embedded spreadsheet, recognized it needed the Excel skill’s rules before touching it, recalculated the workbook, and passed all eight checks on its own ... all without further interaction from me, the user.

A general-purpose skill for editing PowerPoint files has to work for every PowerPoint file, and most of them don’t have a spreadsheet hiding inside a slide.

Writing a Process that anticipates every kind of file that might turn up inside a .pptx archive would be an enormous undertaking, and most of it would go unused most of the time.

A narrower skill doesn’t have that problem, though. A skill built for one job, updating a currency rate in a table embedded in a slide, already knows the two file types are going to collide. Whoever writes it can put the Process in from the start: unpack, recognize the embedded workbook, recalculate it, repack, and flag that the preview image on the slide won’t update on its own.

I recommend playing around with customizing default skills, even the one’s that come from model developers.

The general skills are built to cover the most territory possible, which is a different goal than covering your specific, recurring task well.

Information types make a narrow skill trustworthy rather than just short.

Making sure each each block of text has a clear purpose is easier to get right in a skill built around one real task than in a skill trying to cover everything a file format could ever contain.

That’s one of the take aways from this research, and I hope this gives you a more detailed view of why.

➡️ Check out this free information type decision guide, if you want to dig more into that.

As you may know by now, agent documentation is the critical backbone of any AI content process or knowledge management.

Stay tuned for more explorations into how AI works with structured documentation and knowledge bases.

➡️ Paid subscribers get full access to my Structured Skills Library — the same information-type discipline used to diagnose this skill, applied live across every skill I use in my own writing and teaching workflows.

➡️ If you want to build that discipline into your own workflows, my course Writing with Machines walks through designing AI workflows for content creation from the ground up.

I would love to hear what skills you are working with in the comments!

Subscribe now

Unsorted content gets chunked arbitrarily by AI, and the structure you built vanishes the moment it’s uploaded. This free guide covers the five information types and a decision procedure for typing any chunk, so your content stays legible to readers and AI alike.

We’ve all heard it by now ... garbage in, garbage out. But what does that exactly mean?

Give a model a pile of unsorted notes and it will impose its own order on them. The output can read confidently, but still not sync up with your context and knowledge.

Sorting your content by what it does before the model sees it is often a better fix than a prompt or model upgrade.

I’ve been talking about information types for a while (and using them to structure agent documentation). But I realized I haven’t shared an in-depth reference that shows exactly how I’m currently implementing them.

This is the guide I actually use. It covers the five types, the decision procedure, and the places everyone gets confused (including AI). It’s free, and there’s a PDF at the bottom you can just keep.

Here is the core rule: One chunk does one job for one reader need.

If a chunk seems to fit two types, it is usually two chunks. Split it before you type it.

I find that even AI has trouble with this. So I keep updating the reference in the agent documentation I write.

(Paid subscribers can keep up with updates in my public skills library.)

Subscribe now

Where this comes from

This framework is sixty years old. In 1969, Robert Horn pointed out that content has a structure independent of its topic. How you organize a piece of information depends not on what it’s about, but on the job it does for a reader.

His method, Information Mapping, ran through technical writing and instructional design for decades. I’ve written about how Horn accidentally described how ChatGPT would work and what information actually is if you want the longer version.

Precision Content teaches this framework today, and I use Rob Hanna’s five types as the foundational guide for developing structured content and agent documentation.

What has been a discipline for writers turns out to be exactly what makes knowledge legible to an AI model.

The five types at a glance

Every chunk does one job for one reader need. Five types cover those jobs, and each one answers a different question.

The decision procedure

Run the tests in order and stop at the first Yes. The order matters because it catches the easy-to-identify forms first and leaves Concept as the default.

If two tests both say Yes, the chunk is mixed. Split it and type the pieces separately.

The confusing pairs

These four collisions cause most of the errors.

Process vs. Task ... who acts?
Ask whether the reader performs the steps. If you can rewrite each step as a command the reader executes (”Click Save,” “Enter your ID”), it is a Task.

If the steps only make sense in the third person (”The system validates the request,” “The bill moves to committee”), the reader is watching a flow they do not control. That is a Process.

“How the approval workflow moves” is Process; “How to submit a request for approval” is Task.

Concept vs. Principle ... describe or prescribe?
A Concept explains and you cannot violate it. A Principle guides action and you can follow or break it. The test is whether someone can disobey it.

For example, “A style guide defines house conventions” is a Concept (a description). “Use sentence case for headings” is a Principle (a rule you can obey or ignore).

Concept vs. Reference ... understand or retrieve?
Concept is prose you read once to grasp an idea. Reference is data you return to and scan for a specific value.

If it is a definition written to be understood, it is Concept. If it is a lookup entry, table row, or spec you would never read start to finish, it is Reference.

A paragraph explaining what an API key is → Concept. A table of key names, types, and defaults → Reference.

Principle vs. Task ... rule or steps?
A Principle states what should be true and is not sequenced. A Task is ordered and performable.

“Always back up before upgrading” is a Principle. “1. Export the database. 2. Run the upgrade. 3. Verify.” is a Task.

A Principle often explains why a Task step exists, but it is a separate chunk.

Signals to look for

Cues a person or a model can detect in the text itself. Treat them as evidence, not proof. Confirm with the decision procedure.

Worked examples

The decision procedure, applied to real chunks. Read each one, then check the type and the reasoning.

Common mistakes

Mixing description and instruction in one chunk. A paragraph that explains what something is and tells you how to do it is a Concept and a Task glued together. Split them.

Calling a Process a Task because it has numbered steps. Numbering isn’t enough. Ask whether the reader performs the steps or watches them happen.

Calling a Principle a Concept because it sounds explanatory. If the reader could obey or violate it, it is a Principle.

Describing the tool instead of naming the concept. The Concept slot pulls toward “this skill produces…” or “this report shows…”. That is purpose, which belongs in a description or the Task.

Concept names the thing the reader must understand to make sense of the rest. If the subject of the sentence is the tool itself, it is probably a description, not a Concept.

Burying Reference data inside prose. A specification, threshold, or list that readers will scan for one value belongs in a table or list typed as Reference, not in a sentence.

Typing a whole document as one type. Types apply to chunks, not pages. A single page usually contains several.

When in doubt

Ask what the reader is trying to do with the chunk:

• understand it (Concept)

• look something up (Reference)

• decide or judge (Principle)

• follow how something works (Process)

• perform steps (Task)

The reader’s need settles most cases the surface features leave open, right?

Why this matters for AI

AI reads content in chunks, not documents.

When your content is typed, the retrieval system finds the right unit, and the answer is grounded in your actual work. When it isn’t, though, the system chunks arbitrarily, and the structure you built vanishes the moment it’s uploaded.

I’ve written about reading with information types and using information types to build and evaluate prompt structures if you want to see the method in practice.

There’s also a worked example of information typing on a real source.

The PDF

The same guide, formatted for printing or keeping on your desk. Download the Information Type Guide (PDF)

The agent-ready version

If you want the reference version formatted for AI systems to read, with the same decision procedure written so a model can apply it, it’s in my Structured Skills Library ... alongside the skills I use to build and maintain a typed knowledge base.

I had five years and 218 Substack posts, exported as a pile of HTML files but nothing you could actually query. Here’s how I turned that archive into a real knowledge base, using structured approaches like taxonomies and batch processing.

If I’m honest, I’m not really someone who organizes as I go.

When I finally decided to try building my own LLM wiki as a folder of markdown files, I had accumulated 218 Substack posts.

(If you are new here, an LLM wiki is a simple, but organized knowledge base that you connect directly to your AI.)

Whenever I needed to reference or find an old post, I generally scrolled one post at a time on the Substack dashboard.

Of course, I always have a sense of my past writings. But without scrolling for twenty minutes, I could not tell you which post first made a given point, or whether I’d already made it three times under three different titles.

It is a problem that’s plagued me and sent me off looking for the perfect app.

I’ve written about the graveyard of note apps behind me ... Mem, Notion, Roam, Anytype.

Each one solved a different piece of the problem and then stalled out, because storage, editing, and AI access were all bundled into one product I’d eventually outgrow.

The fix was to stop treating those as one job. Separate the three layers, and you can swap each one as your workflow changes:

• Storage — a folder of markdown files.

• Structure — an editor that reads the files without adding its own syntax.

• Access — an AI pointed at the folder instead of owning it.

That was the theory. Substack was the first real test of it at scale.

Subscribe now

How I exported and organized 218 Substack posts

So I pulled all 218 posts off the internet and rebuilt them as a knowledge base.

Not a backup. Not an archive. Something an AI could actually query with precision.

The difference between a folder of old posts and a working wiki is more about what happens before you import anything.

At first, I thought this would be simple.

Substack has an export button, so I clicked it, downloaded the zip, and opened it up expecting at least text files that were ready to go.

Instead, all I saw was a bunch of HTML files. One per post. Each file still had web page artifacts like subscribe buttons and pull quote widgets all in markup.

Now I’m not a coder, and writing a script really isn’t my thing. That’s where Claude Cowork came in.

I described what I had and what I actually wanted, and it wrote a script that stripped the Substack widgets out first, then ran the real HTML-to-markdown conversion on what was left.

But stripping the HTML only got me markdown files. It didn’t get me a knowledge base. Just 218 loose text files sitting in a folder, no better organized than before.

So before I let anything touch the real vault, I had to decide what I was actually building.

• What counts as a domain.

• What tags mean

• What goes in the frontmatter of every single file, so an AI reading one of these posts later knows what it’s looking at or for.

Skip that step and the import is just digital hoarding with better formatting.

It just so happens, that I’d been reading Manny Silva’s book, Docs as Tests with AI, around the same time, and one idea stuck.

Don’t just trust that a piece of documentation does its job. Test it. Applied to 218 posts, that meant not dumping them all in at once and hoping everything got organized correctly.

Do them twenty at a time instead.

For each batch, Claude proposed a domain, a type, a set of tags, and handed the whole thing back as a table, one row per post that I could scan top to bottom to catch any major mismatches.

It was slow going for 218 posts, but that’s part of the point. A wrong classification, once it’s live, is worse than no classification at all, because errors simply scale with the knowledge base.

Once I approved a batch, Claude Cowork took the next step. A script picked up the approved table, wrote the frontmatter into each file, and moved it into its folder.

I checked the work, not every individual move.

Mapping content from the ground up

Doing this helped me better understand the nature of my content and how to build organization rules.

In a way, domains started clustering on their own.

I asked Claude to build a Map of Content for one of them, ai-and-writing, and it pulled ten clusters straight out of the classification tags. No hand-curation. All 91 files in that folder linked exactly once. Zero orphans.

To actually become a wiki, the content needs to be able to connect beyond just the folder structure. A MOC isn’t just a table of contents, but an actual map for important topics or key words that connect a level above folders.

I liked those MOCs enough to turn a few of them into their own Substack posts that point readers back through five years of writing organized by theme instead of by publish date. Something I’d been meaning to do for a while, and kept putting off, because doing it by hand is tedious.

None of this would matter if the schema only helped me look backward, though. What happens the next time I sit down to write something new?

I finished a draft a few weeks ago and ran it through a skill that scans a near-finished piece for link opportunities before it goes out. It doesn’t touch the writing itself. It just reads the draft, checks it against everything I’ve published before, and reports back where a backlink might be added.

It found three without me having to go digging anywhere. Every one of those 218 posts carries a published_url and a real domain and type in its frontmatter now, the same fields the batch tables assigned back when I was still checking classifications one table at a time.

The metadata I built for the import is what the skill reads to make the connection now.

That’s the actual payoff for all this. Not a tidier folder. A knowledge base that can cite itself and project to readers a more coherent body of work.

That skill is already up in my public Structured Skills Library, if you’re a paid subscriber and want to go pull it apart right now. I’m also sharing it directly here soon.

If you’ve got years of writing scattered across a platform’s servers, the export is the easy part. Substack will hand it to you in five minutes. The work is deciding what you actually want to be able to ask your own archive later.

Below is a short reference that paid subscribers can use to start this process themselves ... and of course, more specifics can be found in the Structured Skills Library.

Read more

Four papers published this year measure whether agent skills actually work, and they all complicate the common advice about skills. It’s not necessarily how long the skill is. Here’s what works and how to spot what doesn’t using structured approaches and information types together.

I’ve spent this summer building a structured knowledge base — not just files I upload into AI, but one that helps me manage and organize them.

This has been the ghost I’ve been chasing, mostly through a graveyard of personal knowledge management apps. Every single one ended up in chaos.

This year I stopped looking and started building my own instead.

One working piece of that is the pipeline connecting my knowledge base to Blue, the project tracker I schedule this newsletter through. I published the skill behind it in Context Lab a few weeks ago. It’s called cyborgs-briefing.

This week, though, I was deep in the unglamorous part — revising those skills, reorganizing the documentation, and figuring out which had quietly gone stale.

Subscribe now

What is a skill?

If you haven’t worked with these, it probably sounds more complicated than it actually is.

Basically, an agent skill is a plain markdown file, usually called SKILL.md. It sits in your folder and tells an AI agent how to do one specific job. You can also install them directly into AI tools like Claude and ChatGPT.

The format is simple and doesn’t really change across AI models.

A few lines of metadata at the top say what the skill is called and when it should be used. Below that are the instructions in natural language. A skill folder can also hold supporting files:

• scripts the agent can run,

• reference documents it can pull from,

• templates and examples.

But the SKILL.md file is the only required part.

What makes it different from a prompt is persistence and scope. A prompt is something you type once for one conversation. A skill is written down once and read every time the job comes up, by whatever agent happens to be doing it.

That makes it documentation, with all the maintenance problems documentation has always had.

Years ago I wrote about the anatomy of a prompt — how a good prompt separates task from context from content instead of mashing all three into one paragraph. A skill works the same but at a larger scale.

How my skills actually fit together

Here’s my drafting process, simplified:

1. A draft starts. Sometimes I write it from scratch, sometimes I ask AI to draft from my notes. Often it’s both.

2. I iterate on the draft. Often section by section or by task.

3. Skills run passes on it. One audits for AI-sounding language. One scans for missing links or citations. One polishes the opening and writes the AEO summary.

4. I revise by hand. This customizes the text. I also track my revision habits and use that to help AI improve its revision skill.

5. Publish. This is more than just posting on the web. I have AI incorporate meta-data and then organize it into my knowledge base.

But as I continue to fine tune how skills support this process, I’ve been looking for research on what actually makes a skill good.

The research has interesting findings, but there’s a blind spot.

What the research actually found

Li and colleagues’ SkillsBench is the most direct evidence available.

They tested 86 real tasks with skills mounted versus without. Average performance improved, but here’s what matters: agents cannot reliably write their own skills.

When models tried to generate skills for tasks they’d been given, performance dropped. The skills need to come from someone else — someone who understands the task deeply enough to document it.

Let’s call that person a writer.

More generally, the benchmark shows three patterns:

1. Skills can backfire. Some tasks got worse when a skill was attached. The skill prescribed a single heavyweight process with no way to opt out. It fired every time, even when it shouldn’t. A skill with no off switch can do more harm than no skill at all.

2. Focused beats comprehensive. Skills that tried to cover every edge case and scenario underperformed. Tightly scoped skills, with constraints clearly marked, outperformed ones that attempted to handle every variation.

3. The agent has to sort what it’s given. More guidance doesn’t mean better performance if the guidance is tangled. The agent has to separate rules from steps from examples before it can use any of them. That sorting is overhead.

Wu and Zhang frame skills as procedural knowledge: knowing-how, a different kind of content from knowing-what. And Jiang and colleagues formalize a skill as four parts:

• a condition for when it applies,

• a policy for what to do,

• a condition for when it’s done,

• and an interface so something else can call it.

Ling and colleagues took a different angle. They looked at what people actually publish: 40,285 skills in a public marketplace, heavily concentrated in software engineering. What they found was “widespread intent-level redundancy” — the same capability, rebuilt over and over under different names.

You need writers, not just models

In the end, agents can’t generate their own good skills. When a model tried to write a skill for itself, performance tanked. Skills need to come from a human who understands the task deeply enough to document it carefully.

The bottleneck isn’t the model—it’s you, the writer. And you need a method.

Every finding above is a symptom of the same underlying problem, and the papers keep describing it with the wrong instrument.

SkillsBench measured length. Not because length is the actual problem, but because that’s what you can measure automatically. But the real issue is five different kinds of content get stacked into one file.

That organization is work an actual human writer should have done.

I’ve argued before that a skill is documentation, not a prompt. Documentation has a methodology for this, built over decades. Content gets sorted by the job it does for the reader. This is how writers have always organized complex information.

I use mostly five types.

When I restructured my own pipeline, I applied this grid to every skill.

The cyborgs-briefing skill that orients a drafting session is pure Concept and Task, which explains what a briefing is, then walks through the steps.

The structured-note skill I published this week sorts incoming content by type before it goes into the knowledge base — reference notes, concept notes, principle notes, and so on.

Both work because every block has one clear job. Both fail instantly if you mix the types without thinking about how they all connect.

Under this lens, all three failures reported in this research can be traced back to missing Principles.

Jiang and colleagues describe a skill as needing an entry condition (when to start), a process (what to do), and an exit condition (when you’re done). Missing that exit condition? The agent keeps working past “good enough.”

That’s a missing Principle. There’s no rule for when to stop.

The thirteen tasks that got worse? Same thing.

The skill had no boundary. There was no Principle saying “don’t use me if X.” So it fired every time, even when wrong.

And Ling’s marketplace redundancy happens when Principles aren’t separated from Process. Every small variation in the rules spawns a whole new skill file instead of an update to a shared one.

Manny Silva arrives at the same place from the practitioner side. I reviewed his book, Docs as Tests with AI, here in May. He argues that a skill needs three things — entry criteria, process steps, and an output specification.

He also argues that documentation should load in layers.

1. Metadata first — so the agent knows if this skill even applies.

2. Instructions next — what to actually do.

3. Resources last — details you only need if a step calls for them.

Separating these out into organized chunks helps both humans and readers understand and implement these skills.

His framework maps neatly to the five types.

• Entry criteria are Principle.

• Process steps are Task.

• Output specification splits into Reference (the shape) and Principle (the standard it must meet).

I would add to this anatomy a Concept type, making sure the AI understands key ideas about the task that are contextual and may be different from its training or what it finds in RAG or on the web.

Tacit knowledge is one of the important things the benchmarks can’t measure.

In his book, Silva describes a skill built from everything a practitioner consciously documented about her process … and it still failed.

She performed one comparison step so automatically she never thought to mention it. No amount of type-checking catches what you don’t know you know. That takes testing against real work.

Try it yourself

Try this the next time you are working with a skill. Go through your skill draft and label every block with the job it’s doing.

• explanation (Concept),

• lookup data (Reference),

• a rule (Principle),

• a flow (Process),

• or a step (Task).

Anything that doesn’t fit one of those five is either in the wrong place or shouldn’t exist. Anything that mixes two types should be split.

That’s it. No length audit needed. No word count. Just ask what job each block is doing.

Next: Inside a failing skill

Now that we have the framework, the real question is simple. What does a failed skill look like up close?

I pulled a real one from the SkillsBench benchmark that lost 13 percentage points of performance.

In the next piece, I’ll walk through that skill block by block, showing you exactly where it breaks. Understanding the problem is one thing. Seeing it in action is another.

I would love to hear what skills you are working with in the comments!

Leave a comment

Sources

• Xiangyi Li et al., SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks (arXiv:2602.12670, 2026)

• Yanna Jiang et al., SoK: Agentic Skills — Beyond Tool Use in LLM Agents (arXiv:2602.20867, 2026)

• Yaxiong Wu and Yongyue Zhang, Agent Skills from the Perspective of Procedural Memory: A Survey (TechRxiv, 2026)

• George Ling, Shanshan Zhong, and Richard Huang, Agent Skills: A Data-Driven Analysis of Claude Skills (arXiv:2602.08004, 2026)

• Manny Silva, Docs as Tests with AI — my review

• citation-check task and citation-management skill, benchflow-ai/skillsbench (GitHub)

Sorting what goes into your knowledge base isn’t the same as sorting what it actually is. This post walks through the structured-note skill I use to turn a bullet journal line, a voice memo, or a half-read article into one note typed by information type (concept, reference, principle, process, task) with a real worked example and the skill itself to try.

Throw a handful of mixed seeds onto bare dirt and something will definitely come up. You just won’t know what, where, or whether it’s anything you actually wanted.

My markdown knowledge base isn’t run that way. I don’t dump everything from the notebook into it and trust AI to sort out what grows.

It’s closer to a garden. Every note gets sorted before it goes in the ground, placed where it actually belongs, and given the conditions that kind of knowledge needs to take root.

Last week I wrote about why a $12 notebook beats every AI note app I’ve tried at separating storage, structure, and access. I promised I’d come back with how I’m building the digital half of that system.

“Digital gardening” is a common term for this kind of bottom-up approach, and like any kind of gardening, requires the skill of identifying and sorting what goes where.

Not everything from the notebook goes into the markdown knowledge base. Every knowledge base needs a filter. But that’s only the first step.

Before a note goes in the ground, it has to be sorted by what it actually is, not just what it’s about. That’s the job of what I’ve been calling the structured-note agentic skill that takes something I’ve written, said out loud, or read, and turns it into a note organized by the work each piece of it is doing.

In case you’re a newbie, here are the five core information types:

• A concept explains what something is or why it matters.

• A reference is a fact you look up and move past.

• A principle is a rule you can follow or break.

• A process describes something unfolding that you watch rather than perform.

• A task is steps you actually carry out.

I’ve written about this framework before, so we’ll keep the brief so we can get on with the structured skill.

Subscribe now

But those are certainly not the only information types, especially if you gather knowledge about more than just documentation. For example, many things I research make arguments built on specific warrants. You can get even more granular with claim, grounds, warrant, backing, qualifier, rebuttal, counterclaim. The basic rhetorical building blocks of most arguments.

The other addition is metadata like purpose, audience, tags related to my work, etc. It’s not required to get the benefit of typing your notes, but it’s what lets an AI — or future me — find the right note without rereading everything first.

To help make this process more efficient ... because, yeah, it can be a lot of work ... I created a structured-note skill.

This is the skill that shapes a note before I plant it. Sometimes it’s a line in the bullet journal. Sometimes it’s something I talked through out loud into Pocket (referal), my voice-note app, on a walk. Sometimes it’s an article sitting in Readwise Reader, highlighted and half-forgotten.

A few months back, all three fed the same note. I’d jotted a line in the notebook about breaking big goals into short “Sprints,” talked through the idea into Pocket, then finally sat down with Ryder Carroll’s piece on the same idea and ran the whole thing through the skill. Here’s a condensed version of what came out:

## Reference: what a Sprint is and the three requirements

A Sprint is a short, self-contained project covering a subset of a
larger goal. Unlike a phase, a Sprint is an end in itself — it
produces its own satisfaction, information, or motivation, rather
than existing only to lead to the next step. A Sprint has to have
no major barrier to entry, break down into clearly actionable
tasks, and stay under a month, ideally one to two weeks.

## Process: how Sprints sustain motivation

A long-term goal pursued all at once behaves like a marathon — the
longer it takes, the more it drains patience, and drained patience
is what causes projects to get abandoned. A Sprint interrupts that
drain with frequent, visible checkpoints of progress.

Claim: Sprints protect motivation better than open-ended goals.
Grounds: one writer lost motivation when his task app erased each
item on completion, leaving nothing to show for finished work;
switching to a typewriter, where pages physically piled up, made
progress tangible again. Warrant: visible, accumulating progress
is what sustains motivation on projects with no external
accountability.

## Principle: rules for keeping a Sprint workable

Keep every Sprint under a month. Estimate how long it'll take,
then triple that number and plan against the tripled one. If you
don't have enough time to meet even the tripled estimate, shrink
the Sprint's scope — don't let the timeline slip.

This pattern carries throughout the note, recording how the Sprint cycle actually unfolds, the steps for setting one up.

That middle section used to be typed as Concept, not Process, until I ran it back through my decision guide. These distinctions are harder than they look, and AI gets them wrong too, which is the whole reason I built the guide in the first place. It’s in the public skills library if you want to try catching your own mistypes or use it for a reference for AI.

Now I can more clearly find the specific kind of information I need ... but so can AI.

If you’re interested in trying the skill yourself, all my skills are available in my public skills library for paid subscribers. Fully adaptable, and I’d love to hear what changes you’d make. I’m also sharing an installable bundle in that library, so you can plug and play if you just want to take it for a test spin.

That’s the planting done. What’s still ahead is the garden itself, for example how the notes actually sit together once they’re in the ground, what an entity is versus what an information type is, and what I’ve started noticing growing in there that I didn’t plant on purpose.

I’ll get into that next.

Curious what the sorting actually looks like under the hood? Below is the skill itself for paid subscribers, copy-paste ready.

Read more

Most AI note apps bundle storage, structure, and access into one product and get at least one wrong — a $12 bullet journal doesn't. Drawing on structured authoring and Ryder Carroll's method, this piece shows how to separate what stays in the notebook from what graduates into a structured, AI-searchable knowledge base.

The best PKM I’ve used doesn’t sync, doesn’t have an AI layer, and has never been acquired or sunset by a startup out of money.

It’s a $12 notebook ... and it solves the three problems that I wrote about in my last piece on knowledge management (and what most apps get wrong): storage, structure, and access.

I know I promised a deep dive into the anatomy of a skill not too long ago, and honestly, it was scheduled for today.

But work I’ve been doing with my bullet journal turned out to be more relevant to how I’m thinking about personal knowledge management (PKM) right now, so that piece is back in the queue.

In fact, I’d argue that if you actually want to understand how a PKM works, or how a skill should be structured, you have to start at the most basic level there is ... analog.

I don’t write about this much here, but a notebook has been my first-layer PKM tool off and on for years, specifically the bullet journal method.

I slipped away from it for a stretch, chasing what I was sure would be the perfect AI-powered PKM instead.

I wrote a Substack note about that particular rabbit hole, but it turned out to be more relevant to my explorations than I thought.

These days I’m using my bullet journal as my launch pad for experiments with local markdown knowledge bases for AI (which is where all this is going!).

Prepping for a bullet journal community panel a few weeks back is what got me thinking about this connection again.

But wait ... I’m not talking about the versions of bullet journal with stickers and washi tape you see all over Instagram. I mean the original method, the one Ryder Carroll actually designed.

If you’re looking for a PKM that gets all three parts right, start with this kind of bullet journal.

I spent last week’s piece cataloging every AI note app I’ve adopted and abandoned and how the root of the problem was my inability to control and fine tune the three layers of knowledge management:

• Storage, or where it lives.

• Structure, or how it’s organized.

• Access, or how AI actually reads it.

Almost nothing on the market treats them as three individual units.

If you think about it, analog has been getting this right for centuries ... and is probably why it sticks around no matter how far technology advances.

I teach and research professional writing at a university, but the part of my job that’s actually shaped how I keep that notebook has less to do with the classroom.

For the past several years I’ve worked in structured authoring, a field that mostly lives inside technical writing and enterprise content teams who write documentation for medical devices, aircraft manuals, software APIs, etc.

Many of you reading this newsletter already know what structured authoring is. If you don’t, that’s normal. It’s not really a productivity philosophy, but a way to think about how we build and use information.

Share

What is structured authoring

For those of you unfamiliar, here’s what that looks like in practice.

An aircraft maintenance manual has to hold a torque spec for a bolt, a warning about what happens if you skip a step, and the numbered procedure for replacing a part.

Those look like they could all live on the same page in paragraphs, and in a bad manual, they do.

But they’re three different kinds of information, doing three different jobs for the reader.

• The torque spec is something a mechanic looks up and moves on. It’s a fact, and it shouldn’t change based on context.

• The warning is a rule. Follow it or don’t, but it’s not a step in a sequence.

• The replacement procedure is something the mechanic actually performs one action at a time.

Collapse those three into one undifferentiated block of text, and the mechanic reaching for a torque value in the middle of a repair has to read past a warning and a procedure to find it.

Keep them typed separately, and each one is exactly where it needs to be the moment someone needs it.

Most PKM approaches rely on storage schemes.

They tell you which folder or list something belongs in. They don’t ask what the thing actually is.

Knowledge items have a purpose. A fact is something you’ll look up later. A rule you’re supposed to follow. A step is something you’re going to perform.

That’s where structured authoring begins. Knowing what kind of information something is will tell you where it goes. You don’t need fancy acronyms.

Long before I had language for any of this, I was already doing a rough version of it every morning with a pen in my bullet journal.

Bullet journal as a structured authoring tool

Ryder Carroll doesn’t call bullet journalling structured authoring.

But he built the notebook around structural moves that map onto three key elements of structured authoring and information design.

A single source of truth. Carroll’s rule is that everything lives in one place, in order, as it happens ... not scattered across sticky notes, apps, and the backs of receipts.

That’s the same principle behind a single-sourced content repository for an airplane mechanic or software company.

There should be one authoritative location for information, so nobody’s ever working from a stale copy or hunting across three systems for the current version.

For us, the notebook is the repository. Its authority comes from being the only place you go to first.

That’s the storage layer that I wrote about in my last post.

Modularity. Carroll compares the bujo system to a Lego set built from discrete pieces you can rearrange without rebuilding the whole thing.

The clearest version of this is rapid logging, where every entry gets marked with a symbol before you write it, and that symbol is really just an information type:

• a dot for something you’ll perform,

• a circle for a fact tied to a moment in time,

• a dash for an idea you’re not ready to classify yet.

Three different jobs, sorted at the point of capture instead of after the fact ... the same way you would build an airplane mechanic manual.

That’s the structure layer part of the knowledge system.

Nothing in the notebook is readable by anything except me until I decide otherwise.

There’s no export button pointed at a model, no ambient permission sitting open by default.

That’s the access layer. You control who and what sees that notebook.

Iterative assembly. Carroll didn’t design the system top-down. He built it one challenge at a time.

Structured content systems get built the same way, when they work. Nobody architects the whole taxonomy in advance and gets it right the first time.

The bullet journal is essentially a structured authoring system that happens to run on paper, assembled by someone solving his own problem without ever needing the technical vocabulary for what he was doing.

It also happens to get the hardest of the three jobs right, by refusing to solve it prematurely.

What this changes about how I actually use it

Knowing that the notebook is already doing this work changes how I use it. Most of what lands on a daily page never leaves.

A task gets done or migrated.

An event passes.

A note gets read once, maybe referenced for a week, and then its job is finished.

The bullet journal is built for things that are useful now and stop being useful soon after, which is most of what a day is made of.

It takes effort to move something from your bullet journal to a personal knowledge management system, so it better count.

This means it is something worth finding and using again some day. When that is the case, I ask myself three things:

• Who’s actually going to read this again?

• How will that “who” find it when they come looking?

• Is this settled, or still moving (something I need to review)?

A note that’s personal, dated, and closed the moment I write it fails all three on purpose, so it stays exactly where Carroll put it, unread by anyone else, human or AI.

A note that’s an idea I’ll want back, findable by topic instead of by day, passes the test and gets pulled out into something more durable.

This is what I call a structured note, filed by what it is rather than when I happened to think it.

That’s also where my tool choices start making sense instead of looking arbitrary.

The notebook stays paper, on purpose. It’s fast, it has no sync delay, and it never asks me to decide anything before I’m ready to.

The structured notes live as plain markdown files I control, not in a fancy proprietary app or format.

(If you don’t know what markdown files are, they are simply text files that have simple formatting markups. It is the most durable and machine-readable format.)

I draft and edit in Obsidian, because it’s really just an interface for the text files. I can move and manipulate my markdown files without working with a third party.

And finally, I use Claude to help maintain the structure once it exists. AI agents help me flag when a tag has drifted, when a note’s type doesn’t match its content, or when something’s been sitting unclassified too long.

(You’re not tied to any of these tools if you are building a knowledge base in Markdown. More on that to come as well.)

Each tool is doing one job, in one layer, and none of them is trying to be the single place everything lives.

The notebook already is that place where the first draft of everything exists.

The five jobs a note can do

You may still not be sure what I mean by information typing beyond bullet journal’s simple bullets.

I’ve written about this framework in more depth before, in Reading with Information Types and What Information Actually Is, but here’s the short version that will get you thinking about what kinds of information are in your knowledge base (not just where).

When something graduates out of the notebook, I don’t just decide it’s worth keeping. I decide what job it’s going to do for me later, because that’s what determines how it gets written and where it lives.

There are five jobs a piece of writing can do, and almost everything I’ve ever wanted to save falls into one of them:

A concept explains what something is or why it matters ... the kind of thing you read once to understand.

---
title: What a structured note is
type: Concept
topic: personal knowledge management
---
A structured note is a note typed by the job it does, not the day it was written.

A reference is a fact you look up and move on from ... a number, a name, a definition you’d never read start to finish.

---
title: Obsidian
type: Reference
topic: tools
---
Local markdown editor. No built-in AI layer. My "human interface" for drafting and editing.

A principle is a rule you can follow or break...a guideline, a standard, a “do this, not that.”

---
title: Type and content must match
type: Principle
topic: note maintenance
---
Never let a note's type and its content disagree. If you catch one, fix it before moving on.

A process describes how something happens, when you’re watching it unfold rather than doing it yourself.

---
title: How a note graduates
type: Process
topic: bullet journal workflow
---
A note gets written, sits in the notebook, and either closes for good or gets flagged to graduate.

A task is a set of steps you actually perform, in order, to get something done.

---
title: Graduating a note
type: Task
topic: bullet journal workflow
---
1. Reread the note.
2. Decide if it's settled.
3. Retype it.
4. File it by topic, not by date.

The block above each example is metadata. It’s the advanced version of this move. You don’t need it to get the benefit of typing your notes.

But once you’ve been doing this a while, tagging a note with its type and topic directly is what lets you find it later without remembering a single word of what you wrote.

You’re searching the label. No more guessing at the content.

You can also add context there for AI which helps it use your knowledge better.

Naming the type tells me the shape it needs. A table. A rule. A sequence. A short piece of prose. No more guessing.

There are more types that I’m exploring, but these are the basic ones serve as the core to any knowledge base.

Sounds like a lot of work ...

It isn’t, most days, but it’s worth being honest about why I bothered building it, and the honest answer has changed in the last few years.

I’ve been building a personal knowledge base that an AI assistant helps me maintain, and typing turns out to matter even more.

When I browse my own notes, I can tolerate some mess. I remember roughly what I meant, even when the label’s wrong.

An AI searching the same notes doesn’t have that memory to fall back on. It only finds what it’s told to look for, and untyped, undifferentiated notes give it nothing to search against except raw text and hope.

If the pattern happens to match, it might still work, but it’s much more random than you’d want.

If a note is typed, an AI assistant can tell the difference between a fact I’m citing and an idea I’m still testing, without me having to explain it every time I ask.

So if you plan on integrating AI into your knowledge base, it’s worth the work. But it’s one payoff among several, and I’d have kept doing this even without it.

A typed system is faster for me to search six months later, when I’ve forgotten the exact words I used but still remember what kind of thing I was writing.

It’s easier to hand off to a student, a collaborator, or future-me, who can open a note and know what to do with it without reading the whole thing first.

And it scales as my work grows.

Carroll built a system that already knew this intuitively, years before anyone needed to worry about what an AI could and couldn’t read.

That’s the irony, I suppose. The paper notebook is about as close to a perfect PKM as you can get, because it never tried to do all three jobs itself.

Next, I’ll be sharing how I’m building my local knowledge base of markdown files and how I’m developing the workflows that maintain the system.

The creator economy looks like a new phenomenon of the social media age but technical writers and content strategists have been running the same playbook for decades ... giving away useful knowledge to build trust. This piece traces that history from Robert Horn’s modular content to Ann Rockley’s “intelligent content,” and shows how the original creators also built the structured content today’s AI depends on.

Part 2 of a series. Part 1 asked whether AI is killing your professional association.

AI isn’t the biggest disruptor in today’s knowledge environments.

That’s a case I’ve made informally for a while.

The creator economy is more disruptive than AI, but often goes unaccounted for.

In part one, I argued that AI didn’t kill the Society for Technical Communication ... or any of the other numerous professional associations that are struggling or even gone at this point. For example, STC had been losing members for twenty years before ChatGPT existed.

In this post, I’m going to make a case for what I think has been killing professional associations.

In short, people changed where they went to learn their craft, and who they trusted to teach it.

As knowledge and content has become easier to create, share, and find through the internet and social media, professional communities have been quietly gathering around “creators,” or professionals who create useful content direct to audiences, often for free.

We call it the creator economy, though you may or may not have heard that term.

Associations and institutions that adapt to this new economy survive, even thrive. Those who don’t, disappear.

That’s my theory anyway.

Subscribe now

What is a “creator”

When I say creator, I don’t mean influencer.

The two get lumped together, but they are opposites in a lot of ways.

An influencer grabs attention and rents it out to businesses, often posting about products in exchange for a fee or a discount code.

A creator makes something people actually want, like a newsletter, a podcast, a video series, a course. The audience forms around the usefulness of the work itself, as well as the “ethos” or credibility of the creator.

Ethos is just an old rhetorical word for earned trust. A creator builds it by giving useful work away, not by holding a title or charging dues.

The creator economy is now an entire ecosystem of these independent producers in spaces like YouTube, Substack, Patreon, Linkedin, and many other spaces.

Goldman Sachs sized the creator economy at roughly 250 billion dollars in 2023, with tens of millions of people creating for a living. It has only grown since then.

No single platform holds it together. Instead of going to an institution or an association, people follow a person who has been generous with what they know.

You might think this is a phenomenon of the social media age with YouTube stars and the first full-time bloggers.

But if you look a little earlier, being a creator has been around for much longer.

Content strategy and the creator mindset

Tech writers are creators at heart ... actually all writers are, really. That’s what we do. Create useful content for specific audiences.

That’s most visible in the discipline of content strategy, which is the work of planning, writing, structuring, and maintaining everything an organization publishes so that it stays useful instead of collapsing into a mess.

Kristina Halvorson, who helped name the field, called it planning for “the creation, publication, and governance of useful, usable content.”

Someone has to decide what a company’s help articles, product guides, manuals, and support pages should say, how they’re organized, how the same explanation gets reused across a website and an app and a printed guide, and how all of it stays accurate over years.

That person is a content strategist, and at the heart of every content strategist is a creator.

If you’ve never touched a content team, stay with me. This is really a story about how expertise earns trust now, in any field.

All businesses and institutions need to grapple with how the creator economy changes the ways people engage with information.

Creators before there were creators

Go looking for where content strategy came from and you find something that resembles the creator economy, only earlier and mostly offline.

, a content strategist who wrote the field’s own history, traces its founders, and I would argue that their credentials and association memberships have little to do with their success.

If you are not in the field, you’ve probably never heard of these people. That’s kind of the point.

Ann Rockley, , Kristina Halvorson all wrote self-published books, ran their own consultancies, and started conferences that people paid their own way to attend.

They earned trust the way a creator does now, by giving useful work away until people came to depend on it.

For example, Rockley is called the mother of content strategy because she showed up year after year teaching people how to manage mountains of content. That’s how you build ethos in the creator economy.

Or take Scott Abel. Since 2003 he has run something called The Content Wrangler ... more than a hundred free webinars a year, over sixty thousand subscribers, no membership card and no dues. Just a person who kept showing up with something useful, until an audience gathered around him.

Trust used to be handed to you by an institution, a credential or an association or a byline in the right journal. Now you earn it in public by being useful.

Let’s go even further back.

In the 1960s a researcher named Robert Horn built a method he called Information Mapping. Instead of writing documents as long unbroken prose, he broke knowledge into small, labeled, reusable blocks, so a reader could find exactly the piece they needed.

Around the same stretch of years, Vannevar Bush had already imagined a personal machine for linking and sharing trails of knowledge, and Ted Nelson had coined the word hypertext for writing that readers move through by their own path rather than start to finish.

These were people treating content as something modular and connected, built for an audience to use on its own terms. Decades before anyone uploaded a video, technical communicators and other content professionals were designing knowledge to be reused and shared.

We sometimes called these people technical writers. But they were are just technical writers. That’s why we keep changing the title: information developer, content strategist, content designer, content engineer.

The work keeps outgrowing the word for it.

Sell without selling

Content people have a phrase for the core move used by creators ... sell without selling.

You make something so useful that nobody feels sold to. This could be a troubleshooting guide, a clear walkthrough, a genuinely good explainer. No pitch. The trust does the work, and the sale takes care of itself later.

It’s the whole logic of the creator economy, and it was being studied inside technical communication years before creators turned it into a business model.

In the 1990s a writing scholar named Johndan Johnson-Eilola argued that technical writers had been badly miscast as support staff, when their real work was creating and shaping knowledge, the same high-value work economists were celebrating everywhere else.

More recently, two researchers, Amanda Wall and Clay Spinuzzi, looked closely at how content marketing actually persuades and found that it works precisely by not pitching, for example a set of genuinely useful documents that walk a reader toward a decision on their own.

Content marketing isn’t the same as “content strategy” in the tech writing world, but that’s exactly how many of these influential thinkers have built the communities around their work.

“Give value first” is a creator mantra now ... but a whole profession had it worked out a generation earlier.

Building content for machines

The same instinct that made content people early creators also made them the people who taught content how to talk to machines.

Remember Horn’s modular blocks? I’ve been talking about it for a while. That idea matured over several decades.

By the 2000s it had hardened into a formal standard called DITA, a way of writing content in structured, labeled chunks that software can sort, assemble, and deliver on its own.

Write something once, structure it well, and it can then flow into a manual, a help page, a printed guide, or a search result. Content people call this single-sourcing, and creators reinvent a version of it every time they turn one idea into a post, a thread, and a video.

Ann Rockley gave this the name intelligent content. Content that is both human-friendly and machine-processable. Modular, structured, and rich enough that a machine can find it, recombine it, and reuse it.

Decades before generative AI, a group of technical writers were deliberately building knowledge that machines could read and reassemble. That turns out to be a large part of why today’s AI systems work at all, and it’s close to my own research. Structured content is what makes these tools genuinely useful instead of merely fluent.

It’s also why, when AI writing tools arrived, many content people dug right into testing while other writers braced for a threat.

This is a field already trained to see writing as a system. You design content with users, not only for them, and you work inside a network of templates, tools, and now models that help assemble the final thing.

If your craft already treats writing as something built with technology and other people, a tool that drafts and remixes is just one more part of the network. Another node.

But a language model can be fluent. It cannot earn ethos.

Trust still comes from a person who shows up again and again, giving something useful before asking for anything back.

That was always the real product … and its pretty difficult to automate.

Evolve or die

Everyone will need to adapt to the creator economy before they adapt to AI.

Content people built audiences on free, useful work. They theorized the give-value model of content. They engineered workflows. They spent decades preparing knowledge for machines to read, a valuable asset today that is often overlooked.

Nearly every attribute we now call “creator” was ordinary practice in a field most people picture as the writers of dull instruction manuals.

And it’s not just their story anymore. Every professional is now being asked to do what these people always did ... build trust by giving knowledge away.

If technical writers and content strategists were the original creators, why did the institutions built around them, like STC, fall apart, while creator-style communities in the very same field grew?

That’s the question I’m digging into for my current research project ... and part 3 of this series.

Leave a comment

Substack’s new Pangram AI detector is not a tool that promotes trust ... or even works. Here’s the actual eight-step process behind my writing, and why provenance, not detection, is the question that actually matters.

As many know already, Substack turned on an AI detector last week. I am disabling this tool for all my Substack posts, but not for the reason you might think.

Built with a company called Pangram, this tool lets anyone scan a post more easily and get back a percentage of how much it thinks a machine wrote. It’s on by default, live across the platform now.

I will be turning it off on everything I publish.

Not because I’m worried about what it would find. No one should be surprised that I use AI hard, and I’m not interested in hiding that. I turned it off because I disagree with the premise these detectors are built on (and, honestly, it’s a waste of tokens).

I don’t necessarily disagree with ’s idea of building trust. I just don’t think this tool feeds into that purpose.

I do find the “how I make this” statement an interesting addition, but am disappointed that it’s attached to an AI detector ... making it more of a “defense” than an honest share of workflows.

If you don’t know, this statement is a place to explain your process instead of just submitting to a score. It’s not supposed to be a policing mechanism or some kind of confession, but that’s how it will be perceived when attached to an AI detector.

While I can’t imagine anyone wants to make a habit of reading these, I did think the idea deserved a real post, so I have a single place to point people whenever it comes up.

Then I can get back to what actually interests me ... understanding what happens between writers and machines, not proving I’m on the right side of a percentage that really doesn’t mean much.

For example, I ran a completely AI-generated draft through Pangram’s scanner. It came back 100% human. Then I ran a piece I’d actually written, edited by hand over several passes, sentences rewritten three or four times. It came back 22% AI.

That’s the tool now sitting on every post on Substack.

That said, even a detector that worked perfectly would still be asking the wrong question.

But you should know that it doesn’t really work, anyway.

Why disable Pangram

Here’s the actual mechanism.

On the Publish page of any draft, there’s now a toggle: “Disable AI detection.” Click it, and readers see “AI detection unavailable” instead of a Pangram score when they try to scan the post.

I’ll be flipping that toggle on everything going forward, because I don’t want to support a system built on distrust.

Every scan Pangram runs on Substack almost certainly costs Substack something — a per-check API call to a third-party detection company maybe. Some of that cost gets covered by subscription revenue, mine or my readers’.

I don’t know Substack’s exact arrangement with Pangram, and they haven’t published it, but I don’t want a fraction of a cent of what someone pays to read me routed that way.

For many writing instructors at the university, this is not a new problem.

Turnitin is built on the same premise (even before it rolled out AI). We assume that students are cheating one way or the other and this poisons our interactions around writing with distrust on both sides.

You don’t build a detector for people you trust.

Substack’s value proposition was to build a social space based on human relationship, not algorithms — a network built on people choosing to read each other, not a machine-curated feed.

A detector doesn’t fit into that idea. It quietly replaces it with suspicion as the default posture, whether or not anyone ever gets flagged.

Humans should be the filter, not AI.

How I actually use AI

A detector starts from the assumption that something’s being hidden, and the job is to find it.

I never used Turnitin for plagiarism either, for the same reason. The tool assumes the student cheated and hands you a report to confirm or deny.

That posture doesn’t build trust.

So here’s my “how I make this” statement, offered to you freely, instead of some defense against AI detection.

It’s not really a formula, because my workflows change post to post depending on what I’m writing and how formed the thought already was.

But here’s roughly how this post happened, as an example:

1. I talked out my position on AI detectors into Pocket — a voice-journaling app walking while I thought.

2. That recording became a structured note in my own markdown knowledge base, organized by what kind of thing each part was: an argument, a fact, a story, etc.

3. I developed the thinking further by writing three separate LinkedIn posts responding to Substack’s rollout: one on the platform decision itself, one on the wrong question detectors ask, and one testing the detector on my own writing. Working through those made it clear the idea deserved more room than a microessay, so I decided to compose this post.

4. I had Claude pull up everything I’d already written about AI detectors, back to 2023, so this post could build on arguments I’d already made instead of reinventing them.

5. We built an outline together. Once I approved it, we drafted section by section — each one written, shown to me, and revised before it got committed to a new Markdown file. Nothing moved forward without my sign-off.

6. I ran the draft against a running note I keep of my own stylistic habits when I revise AI text, for example where I break a sentence, when I go conversational versus direct, which constructions I reach for and which ones I’ve banned myself from using too much.
   
   (If you want to see the related documentation, they now live in my skill library.)
   

7. I edited the final wording by hand.

8. Before it goes anywhere near Substack, I run it through a set of finalizing passes: checking for links back to my own previous posts and outside sources worth citing, reading it through the eyes of audiences who might drift or feel left out, and writing the search- and AI-search-facing summary and description that sit at the top of the post.

In nearly every instance I’ve checked, a piece that went through this whole process comes back 100% human anyway.

The real question

The real problem is that we conflate text generation with provenance or authorship. AI slop isn’t slop because there is AI involved, it’s because there isn’t any real authorship.

This is content posted just to post and manipulate the algorithm, not because someone worked through an idea. That’s a human problem that we created long before generative AI existed.

Medium’s Partner Program is a good example. It pays out on views regardless of how those views got earned. That’s created a real AI slop problem.

But nearly every online environment runs on some version of that same incentive: post more, post faster, optimize for the number instead of the thought.

AI just made it easier to do at scale.

Honestly, Substack has been an exception to this … and I think still is (but the AI detector doesn’t really help in any way I can see.)

Provenance is what a detector can’t see. It can’t trace words back to a specific person, at a specific time, making a specific decision to say this or that.

A quote has provenance when you can find the primary source.

A claim has provenance when someone can check it against what actually happened.

Writing has provenance when the person who published it can walk you back through where every piece of it came from, in their own record, dated before the piece existed.

That’s a testable property. A detector’s percentage isn’t.

A piece built for volume has no provenance, whether or not AI touched it.

Nobody wrote it down first, thought about it, decided it mattered enough to say. It was generated to fill a slot, and the slot doesn’t care where the words came from.

Ask me where any claim in this piece came from, and I can point to the dated note it started as in a markdown file I control.

Detecting “AI” measures the wrong variable.

Two pieces of writing can contain an identical percentage of machine-generated text and differ completely on whether either one can be traced back to a person who thought, decided, and stood behind those words.

AI makes this problem more difficult, but detectors won’t solve it.

How I make content (short version)

If you scan this post, or any post of mine, you’ll see it says detection is unavailable.

That’s on purpose.

Here’s the short version of everything above, written to stand on it’s own if this is the only section you read.

I use AI heavily — full drafts, structured outlines, revision passes built to catch my own patterns. It’s a full on agentic writing workflow.

Every idea I publish traces back to a dated note in a knowledge base, so I can show you where any claim in this post came from, not just tell you a machine didn’t write it.

I turned off Substack’s detector because I don’t want subscriber revenue funding a tool built on the same logic as the plagiarism detectors I’ve refused to use in my classroom for a decade.

I have nothing to hide. I just don’t want to fund a tool built for people you don’t trust, and that’s not how I want to relate to the people who read me.

If you want to know how something specific got made, ask. I’ll tell you exactly.

Leave a comment

If any of this made you curious about the actual mechanics (not just the argument, but how the workflow itself works) paid subscribers can take my course, Writing with Machines, free for a limited time.

Or go straight to the source: my live structured skills library documents the specific tools and workflows behind everything described above, updated as I revise them ... because no workflow is static.

Subscribe now

I‘ve spent years switching AI note apps, like Mem, Notion, Roam, Anytype, and each one failed in a different way. That’s because a knowledge base is actually doing three separate jobs: storage, structure, and access. Almost no app treats them as three.

Have you ever bought an app that promised to finally get you organized, used it faithfully for a few months, and then quietly let it slide back into the pile of apps you meant to use?

I’ve done this more times than I want to admit, and each time I told myself the problem was the tool.

It never was.

As exciting as these new apps are, no piece of software does everything you need it to do, perfectly, forever.

Something is always missing, or something almost works but not quite, and eventually you have to connect it to something else, or build the missing piece yourself.

This is exactly why workflows and stacks matter so much right now, for creators and for content professionals both.

Though many will claim otherwise, nobody’s shipping the one tool that does it all. That’s why you’ll often see people build their own combinations instead.

For example, you’ve found the perfect notes app, but handle your tasks somewhere else. Then to get AI to work with both you need to link it somehow. This is why you see so many “extensions” and “integrations” in spaces like Claude and ChatGPT.

Creating “app stacks” that match your workflow like this is a skill in and of itself that is becoming a job in many organizations.

I know tech writers and content designers who make an entire living doing exactly this for organizations, and it’s one of the things that fascinates me most about that world.

What most people don’t realize is that building these workflows is rhetorical, because workflows shape the spaces where we make meaning.

There’s no perfect app or perfect stack sitting out there waiting to be discovered, because choosing and connecting tools always depends on who’s actually going to use what you make, how it needs to move, and what it has to become later.

In the end, the environment you build shapes what you’re able to think, notice, and make inside it, long before AI ever enters the picture.

What AI does is raise the price of getting that wrong. A bad stack used to just cost you time and money locally. Now with AI it costs you at scale.

I’ve been doing a version of this for my own knowledge base for a few years now, exploring various approaches to personal knowledge management, or PKM — a term you may not have heard, but is attached to a category of apps you’ve almost certainly seen. Notion, Roam, Obsidian, Evernote, even a shoebox of index cards if you want to go back far enough are all PKM built to help you capture, organize, and retrieve what you know.

Subscribe now

Three invisible layers to modern knowledge apps

I’ve spent an embarrassing number of hours exploring personal knowledge management, because no single one of these apps has ever done everything I actually need.

The one I’ve been watching the longest, for about four years now, is Anytype — a newer PKM tool most people outside this world have never heard of.

Anytype treats your knowledge as objects instead of notes in a folder — things with real relationships to each other, private by default, but easy to open up the moment you actually want to share or collaborate.

My students use it for their writing portfolios because they can build the structure themselves, and they decide what becomes public.

What it still doesn’t really have is AI. It’s coming. There’s supposedly a working MCP integration now, though I haven’t managed to get it running cleanly on my end yet.

I actually respect how slowly and methodically the team has approached AI, but that can get frustrating at times.

But four years is a long wait, and it’s given me time to notice something else about Anytype. The same object-oriented control that makes it so good is also what makes it exhausting. Deciding what every single thing is, and how it relates to everything else, is real work.

Some weeks I don’t have it in me.

That’s probably the actual reason I haven’t moved my whole knowledge base over yet, more than any missing feature. The AI I’ve been waiting for is exactly the thing that would make that easier. It just isn’t here.

There’s a decent chance I move into Anytype fully once it is. The version of my knowledge base I’ve been building is portable enough that I could hand it to Anytype, or whatever comes after Anytype, without starting over.

But after thinking about it, I realized that this disappointment wasn’t random or even about a single tool.

An AI knowledge base is doing three separate jobs, and almost no product on the market treats them as three or does all three right.

1. There’s storage or where your knowledge actually lives, as files you control, not rows in someone else’s database you’d need an export tool to escape.

2. There’s structure or how you organize, browse, and edit it. Different apps have different approaches that determine what you can do. This is the pillar Anytype nails — and also the one that wears me out.

3. And there’s access or how AI actually reads what you’ve built. It should be swappable, since the model behind it is the piece most likely to change on somebody else’s schedule, not yours. This is the piece Anytype still doesn’t have.

Name the three jobs, and every tool I’ve tried turns out to be strong in one of these, weak in another, and often has no real answer for the third.

My graveyard of knowledge apps

Mem was the closest I came to AI integration, but only by sacrificing the other two..

I imported everything, and for several months I was genuinely impressed. It surfaced things I’d forgotten. Then an update shipped and it didn’t feel the same. I went looking for the knob that I could use to adjust how AI was working.

There wasn’t one.

The organizing scheme was determined by the developers, not me, and there was no way to change it. I ended up building projects in Claude instead, and stopped opening Mem.

I was also an early adopter of Notion, and put my entire life into it for awhile. Within a year and a half it had become a beast I didn’t want to open.

Not because the tool broke, but because the structure I’d built inside it had gotten so tangled that finding anything meant a lot of time and effort that I just kept avoiding.

Notion does storage, structure, and now its own AI layer, all under one roof, which sounds like the dream until you actually try to change just one of the layers.

I also tried Roam. For a while it felt like the answer to everything. I loved that there were no folders at all, just links, with the organization emerging from how ideas actually connected instead of where I’d decided to file them.

I loved it, at least for the first few months.

Then I noticed I was spending more time linking than thinking, more time maintaining the web than using it. There was always one more connection you could add, with the app itself having no opinion about when you’d done enough.

So far the tool that gets it right the most is Twos. I still use it, especially when I’m not sure where to put something.

The structure pillar here is deliberately thin. Twos never asked me to decide much, so I never built anything that could get tangled the way Notion did. You’ve got “lists” and you’ve got “things”. That’s it.

It has AI now and MCP connections that work well, but not many of the organizational “knobs” I might want in a complete knowledge base that I use with AI.

What’s missing in most knowledge apps

But all five tools turn out to be missing the one thing I’ve discovered to be key in developing knowledge bases for AI. Separation of these three layers for better control.

If the AI reading your files is a distinct layer from the files themselves, it can help maintain the structure instead of either ignoring it or owning it outright.

It can catch that two tags mean the same thing.

It can flag a folder that’s drifted from its own rule.

It can follow your instructions on how you want the knowledge base to work.

AI can do the unglamorous upkeep that turned my Notion into a beast and my Roam into a chore.

That’s work I always meant to get around to and never did, and doing it doesn’t require deciding for me what the structure should be in the first place.

That only works because the layer doing that work isn’t the same layer holding the knowledge.

Mem couldn’t offer me this, because it never separated the two to begin with. The AI wasn’t helping me maintain my structure. It was replacing my judgment about what the structure should be.

Anytype could offer me this eventually, once its AI layer actually works, precisely because the object model and the AI are being built as two separate pieces, not one piece trying to do both jobs. That’s the whole reason I’m still waiting instead of leaving.

But in the meantime, I’m building my own personal knowledge base one layer at a time.

1. Markdown files on my hard drive.

2. Obsidian as my “human interface” for drafting and editing.

3. Claude Cowork for maintaining and using the knowledge base.

And I can use markdown documents and skills in the knowledge base to keep guardrails on the AI and help me test and evaluate how the knowledge is working.

More explorations to come

This is the first of several pieces I’m planning to write on this exact problem, working through each pillar in more depth and exploring various structured approaches to building and maintaining an AI-ready personal knowledge base.

I’m already exploring how skills and information types make this workflow work better, as well as different ways of connecting AI to a markdown knowledge base.

Next, I’m pulling one of those skill files apart, line by line, to show what happens when nobody decides what belongs in it.

If you want to see some of that work, I’ve been building a public library of the skills I use for my own writing and content workflows — real, working files, organized by the same five information types I keep coming back to, available to paid subscribers to copy and adapt for their own setup.

It’s a live library, not a one-time download, so it keeps changing as my own practice does.

So ... of the three jobs your knowledge base is doing right now, which one have you never actually chosen? Where it lives, how you’ve structured it, or how AI gets to it?

Leave a comment. I’d genuinely like to know which failure modes you’re living with.

Leave a comment

I spent the summer building a markdown knowledge base that AI helps me manage, then went looking for research that would tell me it was worth it. Four studies across twenty-seven years, including the first real evaluation of AGENTS.md context files, suggest the organizing work is easier to demand than to justify.

I’ve got to be honest with you. I’ve spent most of this summer building workflows around a personal markdown knowledge base that AI helps me manage.

Tags, links, folder structure, naming conventions. All the housekeeping I’ve quietly avoided for twenty years. Everything I hate about managing my own notes is exactly what keeps sending me off to try one more app.

Surely there’s a perfect one out there. There isn’t. I know there isn’t.

I go looking anyway, about twice a year. For a brain that would rather build the system than sit down and use it, this is a dangerous hobby.

I’m Lance Cummings. And welcome to my now monthly podcast that explores deep research on AI and writing.

After all this, I decided it might be worth looking into what kind of research has been done on personal knowledge bases.

Oh, yeah ... there’s lots of creators out there giving out all sorts of ideas and frameworks, but what does the research say?

Here are some of my initial findings.

I looked at four papers, written across twenty-seven years, are all circling the same question from different sides: what does it actually cost to make your own knowledge explicit, and who ends up collecting?

That question used to have an easy answer.

You did the filing for future-you, future-you rarely showed up, and the whole thing quietly rotted. What’s changed is that my knowledge base has a second reader now. An AI assistant searches it, and it searches differently than I browse.

So it sounds like a reason to finally do the work of organizing. And it might be.

But you might want to hold on a second.

The structure nobody wants to do

In 1999, Frank Shipman and Cathy Marshall point out in their article [Formality Considered Harmful] that the real problem is motivation and intention.

After years of building systems people wouldn’t use, the real problem came down to something simpler than interface design. Users were unwilling or unable to make structure explicit.

If asked to declare what a thing was or how it related to other things (👋 information types), people balked, worked around it, or filled the field with garbage.

Not from laziness. The cost of formalizing is immediate and the payoff is speculative and later. You know what you mean right now. You are, at that moment, the worst possible judge of what future-you will need.

This is probably one reason it’s so hard for content strategists to convince stakeholders to structure their content.

Their solution really ties to how I’m working with AI to build a personal knowledge base. Rather than demanding structure up front, systems should offer incremental, system-assisted formalization. The system flags structure you never declared and helps you make it explicit over time.

In other words, build your knowledge from the ground up with help. Start small and use AI to help you discover the organization that’s already there.

That’s more or less what I’ve been building.

It’s also how I nearly lost two folders in my own vault this month, because an assistant had been quietly filing things into subfolders I never asked for.

I got system-assisted formalization, but I wasn’t paying close enough attention to what the AI agent was doing.

What folders and tags were both missing

Jump to 2008. Amy Civan, William Jones, Predrag Klasnja, and Harry Bruce ran a study to find out what kind of organization works best, focusing mostly on tags and folders.

There’s no winner.

Each model has strengths and weaknesses across keeping, organizing, and re-finding information. What study participants needed was a clearer framework for the organization scheme itself.

Think here about things like taxonomies, ontologies ... or simply deciding what goes where, why, and for whom ... and how that “what” gets found.

That’s more important than what tool to use.

What they wanted was a way to say what the categories meant, and to change their minds later without everything breaking. The tool or method doesn’t give you that.

I found this out the hard way.

I counted the tags in my vault a few weeks ago: 1,988 of them. At least forty-five were the same idea written in different ways — “rag” and “RAG,” “pkm” and “PKM.”

There’s a danger that a search or an AI scan might only turn up half the entries for tags like these. The tagging system worked fine. I just had nowhere to record that those two spellings meant one thing or master guide that gives clear guidelines (without guessing in the moment).

That gap is what I ended up filling with three plain markdown control documents: a taxonomy document, tag thesaurus, and workflow overview.

More on that to come in future posts.

How AI is changing the way we think about organization

You’ve probably seen the trend already. To better control agentic workflows, people are starting to do this formalization work voluntarily — not in note-taking apps, but in code repositories, as context files for AI agents, files like AGENTS.md, CLAUDE.md, the README written for a machine.

But there are a few problems arising because developers are mostly controlling what should be called “documentation,” which needs a content specialist.

A 2025 preprint, Agent READMEs by Chatlatanagulchai and colleagues, studied 2,303 of these files across 1,925 repositories. What they found is that these files get maintained the way a build script gets maintained: small commits, constantly, forever.

They grow harder to read as they go. Nobody sits down and revises one.

The content skews hard toward the function ... not context:

• implementation details — 69.9%

• architecture — 67.7%

• build and run commands — 62.3%

• security — 14.5%

• performance — 14.5%

So we tell the machine how to operate the thing, and almost never what the thing is for.

In the content world, we call this the “structure tax.” Maybe you don’t pay a price right now for structuring your knowledge ... but you will later.

That’s why this work is an investment, whether you are a company, a university, or just a solo creator.

The structure tax people refused to pay for themselves, they’re now paying for an agent. [pause]

The part that stopped me

Here’s where I have to complicate my own summer.

The obvious conclusion from all of this is that context files work, so write them. That’s certainly what I assumed while building mine.

Then in February, Thibaud Gloaguen, Niels Mündler, and colleagues at ETH Zurich published Evaluating AGENTS.md, the first rigorous test I know of. They ran coding agents three ways: with no context file, with one the agent generated for itself, and with one a human developer had actually written.

The machine-generated files didn’t help.

They made things slightly worse, and raised inference cost by more than twenty percent. The human-written ones were the only version that came out ahead, and they beat the machine’s version by a clear margin.

The temptation is to have AI generate this documentation for you ... its lots of work. Turns out that could be doing more harm than no context file at all.

But consider this carefully, the agents did follow the files.

When a file named a specific tool, the agent used it almost every time; when it didn’t, almost never. The instructions worked.

What didn’t work was the one thing every official recommendation tells you to put first: the repository overview. It’s the biggest chunk of these files, and it did nothing. Agents found what they needed just as fast without it.

In the end, this is a failure of content strategy. An overview is something the machine can already work out by looking around, but principles and processes needed to properly do that task ... not so much.

The files contained clear information types beyond just an overview. But, honestly, nobody in that study tested a context file designed that way. They varied the model, the prompt, the length, whole categories of content, and none of it moved the needle, because they were all the wrong knobs.

The one variable that mattered was a human deciding what actually belonged in the context file.

Two honest limits to these studies.

Both agent papers are preprints, not yet peer-reviewed. And they measure code tasks in repositories, a long way from retrieval in a personal knowledge base. I don’t think the finding transfers cleanly.

But nobody has run the knowledge-base version of that study, which is roughly where the next several weeks of this newsletter are headed. Stay tuned.

Takeaways for writers and content professionals

To be clear, this does not mean we should “stop organizing your notes.” Rather, we need to pay more attention to how we do it, especially when AI is part of the picture.

So the four things below are the ones I’d still do this week.

Write control documents, not more folders. The 2008 finding is the durable one. Before you reorganize anything, write down what your categories mean and which synonyms resolve to which preferred term. My tag thesaurus took forty minutes and fixed years of broken searches. No new app required.

Audit what your assistant has been organizing for you. If AI helps maintain your notes, run a structure check on purpose, full depth, no shortcuts. Incremental formalization is genuinely useful and it’s also invisible by default. I nearly lost two folders to that.

Treat your context file as configuration, and version it. If it evolves like code, maintain it like code. Small commits, a changelog, and periodic pruning. Files that grow without eyeballs get harder to read and control.

A/B Test your workflows. Run a task with your context file and without it. This is the least fun recommendation but can help you see whether a specific skill or documentation is actually needed..

If you want the practical side of this, the skills that run my knowledge-base-to-Blue drafting pipeline are posted in my public skills library. There are 18 of them up right now, and the number keeps moving.

... and I am updating them everyday, like software.

➡️ The method underneath all of them is what my Writing with Machines course walks through.

Pay it now … or pay it latter

The structure tax hasn’t gone away ... you’ve got to pay it somewhere.

We refused to pay it for ourselves because the payoff was invisible. Now we’re paying it for our machines, and the research says most of what we’re writing down is the wrong half.

Not because structure doesn’t help, but because nobody’s decided what belongs in it and what’s just noise wearing a filename.

That’s the part I can actually do something about. So instead of shopping for a new app this school year, I’m going to build my own, and pay attention to what goes in it.

If you know anyone building content workflows that might be interested in knowing the research behind the work, send this their way.

Leave a comment and let me know how you are using AI for your knowledge base.

I’m Lance Cummings. Until next time — mind what your system is quietly deciding for you.

The Society for Technical Communication, the Special Libraries Association, and other professional groups are shutting down or cutting deep, and AI is the easy explanation. The membership and revenue data tell a different story — one that started twenty years before ChatGPT and explains why some associations are dying while others are growing.

Is AI killing your professional association? It’s an easy question to ask right now.

The Society for Technical Communication (STC) lasted 72 years and had 25,000 members at its peak. On January 29, 2025, it announced its shutdown. Two months later, the Special Libraries Association followed. The Society of Professional Journalists cancelled its annual convention and named a $391,000 deficit. The Public Relations Society of America’s 2026 board chair told members plainly: “We are at a crossroads.”

If you work in tech, you probably heard about STC. If you work in journalism, publishing, library science, or communications, you may have watched something similar happen closer to home. If you’re in a field that hasn’t felt it yet … it may be coming.

The easy explanation is AI. And there’s something to it.

When staying current in your field means creating a prompt instead of attending a conference or reading a gated newsletter, the case for annual dues gets harder to make. But STC’s membership had been falling since 2002. That’s twenty years before ChatGPT. IRS filings show negative net assets every year from 2013 onward.

Whatever killed these organizations, it didn’t arrive in 2023.

I’ve been making a similar argument about higher education.

Assessment was already broken; AI just exposed it. Grades, assignments, and credentials built to gatekeep access to knowledge stopped making sense the moment that knowledge became freely available. Professional associations hit the same problem, just on a different timeline.

I’m researching this now by surveying practitioners across fields in technical communication about what professional community actually means to them, and what they do when the institution behind it disappears.

More on what I’m finding in the posts ahead.

Subscribe now

The numbers tell a slow story

I’d actually been meaning to join STC for a few years.

Every time I got around to it, another no-dues community already had what I needed, so I never signed up. I wasn’t trying to make a point. I just never got around to it. It would seem a lot of technical communicators were making the same quiet choice.

The Society for Technical Communication peaked at roughly 25,000 members in the late 1990s and spent the next two decades bleeding them out. A former board member who tracked the data himself estimates membership would have fallen below 1,000 by the time the doors closed. The organization stopped publicly reporting membership figures after 2007, which tells you something about what those figures looked like.

The finances follow the same arc: revenue fell by more than half between 2011 and 2023, and the organization carried negative net assets every year for the last decade of its existence. The chart below has the full numbers.

What STC was selling made sense when those things were genuinely scarce. You paid dues because that’s how you got the journal, attended the conference, and earned the certification that signaled your competence to employers.

The internet didn’t destroy that value overnight, but it spent twenty years making it cheaper and cheaper to get elsewhere.

Free forums, YouTube tutorials, LinkedIn groups, and eventually entire communities built around specific tools and workflows pulled the same audience STC needed to survive.

AI maybe accelerated the last leg of that erosion, but it didn’t start it.

It’s not just STC

Individual-membership nonprofits that peaked in the late 1990s, built their revenue around dues and an annual conference, and offered credentials and a journal as their core value proposition are under pressure across professional fields.

Some are collapsing. Some are cutting. A few are doing something different.

Two months after STC closed, the Special Libraries Association announced it couldn’t pay its bills either. It had 14,000 members at its peak, a journal, and an annual conference, just like STC. Its members voted to dissolve rather than declare bankruptcy, merging into a larger information-science body by early 2026.

The Society of Professional Journalists cancelled its 2024 national convention, its treasurer telling reporters the organization faced a projected $391,000 deficit. PRSA, the leading professional association for public relations, opened 2026 with its board chair telling members that “meaningful change must occur for us to thrive for another 79 years.”

Not every association is dying. ACES: The Society for Editing has been growing. The Editorial Freelancers Association is growing. Meanwhile, the Association for Talent Development reported $53 million in revenue last year.

What did they do differently?

What this means for your field

So is AI killing your professional association?

Probably not directly.

But it may be the thing that finally made the problem impossible to ignore. That’s the same way it made certain approaches to teaching and assessment impossible to ignore.

What that means for your field, your professional identity, and where community goes from here is what I’m trying to understand.

I’m currently surveying practitioners across fields in technical communication about what professional community means to them now, and what they’ve lost or found since their associations started struggling.

If any of this lands for you, I’d genuinely like to hear from you.

The survey takes about 5-10 minutes.

➡️ Click here to take the survey.

Most skill or prompt libraries are dead the day they’re published. My Structured Skills Library is different: a live, self-updating collection of AI writing and teaching skills, organized by five information types, with copy-paste templates you can adapt to your own notes, tools, and voice.

Too many “prompt libraries” or “skill libraries” are graveyards … a pile of prompts or skills someone wrote once and never touched again.

This library is the opposite. The Structured Skills Library is the live set of skills I use in my own writing and teaching workflows, and it updates the moment I change one.

Each skill is a small program written in plain language, organized by five information types, providing structure for both human and machines to better access, perform and evaluate these skills.

You can read any of them like a short guide, or copy the whole thing and adapt it to your own work.

Available Workflows

As I mentioned, this library is live and constantly changing. As of July 2026, the following workflows and associated skills can be found:

• Content Creation — draft, revise, and publish

• Course Design & Teaching — build courses from scratch

• Vault Setup & Import — bring a scattered note backlog into a structured vault

• Miscellaneous — general-purpose tools for projects and context

There might be more, because I’ll probably forget to update this. (No live connection to Substack. 😆)

How it runs

The library mostly maintains itself. Every skill lives as a markdown file in a knowledge base on my own machine.

When I write a new one, an AI drafts a public version, tags it, links it to its neighbors, and files it in the library.

When I revise the working skill, the public copy updates to match. Every so often I run an audit that checks the whole thing against my own written standards, making sure links, structure, and voice don’t drift.

The result publishes straight from Obsidian, so what you’re reading is never stale.

If this intrigues you and you haven’t yet built your own local AI knowledge base, stay tuned. more to come on that front.

Subscribe now

What you’ll find

• A readable page for every skill: what it does, when to reach for it, and the thinking behind it.

• A copy-paste template inside each page, complete and working, with the personal parts marked so you know exactly what to change.

• Skills grouped by workflow: drafting and revision, course design, and general knowledge work.

• A short guide to the information types themselves, so you can build your own skills the same way.

How to use it

Start with whatever problem is in front of you, for example:

• auditing a draft for AI patterns

• synthesizing your notes

• planning a week of class

• handing a long chat off to a fresh session

There’s a skill for each, and each one tells you when it fits. Read the page to understand it, copy the template when you want to run it, and change the bracketed parts to point at your own files, tools, and voice.

You don’t need my setup to use these. You need your own standards written down, and the library shows you how I wrote down mine.

➡️ If you don’t know what those standards are or how to develop them, check out my course Writing with Machines.

What it costs me

The money side is smaller than people expect, and it doesn’t have to be Claude at all. I run it on Claude Cowork, the desktop tool I use to build and update the library, comes with the Pro plan at about $17–20 a month.

The only other tool is Obsidian, for the notes and the publishing. I do pay around $100 a year for syncing and the publishing tool.

But nothing is locked to one company. That’s what I love about it.

The whole system is plain Markdown files, so you can put any agentic model on top of it.

It already works with ChatGPT’s work mode, and it will run on free, open-source models on your own machine — slower, probably, but free. Conceivably you could do this whole thing for nothing.

(More on local, free setups in a future issue.)

What it actually takes

But the subscription isn’t the real cost. The real cost is thinking.

This library didn’t fall out of the machine finished. It came from a long back-and-forth. I had to redefine what a “skill” even is and making sure information types were correct … something that is oddly difficult even for machines.

The audit and the templates exist because I made the same mistakes often enough to name them.

This is what Manny Silva calls “docs as tests.” You explain your standard, the machine tries, you correct it, and slowly your standard becomes something a machine can actually run on.

The whole thing took me about a day to get up and running, and most of that day went to deciding what my standards were, not fighting the tools.

After that, the work compounds. Once the standard is written down, each new skill is faster than the last, and the machine handles the tedious parts, the metadata and links and consistency checks, while I keep the judgment.

More on all of this to come for everyone, but paid subscribers can check out the library using the form below. ⬇️

Read more

Skills are the backbone of any AI or agentic workflow ... think of it as documentation for your agents (not just a prompt). In this post, I share the actual structured skill I run before every Substack draft, organized by information type and connected to a Markdown knowledge base that makes it work.

Last week I wrote about Slaying the Churnivore with AI Workflows, a visualization of a content plan I’m developing with Claude’s help for my Substack. What I didn’t get into is the layer underneath and thought today ... why not share the skill!

For the past few weeks, I’ve been importing, structuring, and labeling my entire online back-catalog of writing into a Markdown knowledge base connected directly to Claude through Cowork.

This is a knowledge base AI can actually read, navigate, and write back to, because every file in it is labeled the same way I’m about to describe.

That project is too big for one post. Stay tuned. But the skill I want to show you today is a small, working example of what that knowledge base makes possible:

• It can read my guiding content plan directly, and update it when something changes.

• It can review my whole back-catalog to help me think about domains, topics, and where the yearlong arc is going.

• It can find other posts I’ve written and link them into new writing without me searching for them or even remembering they exist.

That last one just happened. While finalizing this post (with a structured skill), Claude linked back to A Skill Isn’t a Prompt. It’s Documentation. without me asking it to look for anything. I didn’t have to remember the exact title. I didn’t go find the URL. The knowledge base already had that information.

A skill is documentation, not a prompt

A skill is a short, saved set of instructions Claude reads before doing a specific job — the same way a process document orients a new hire instead of re-explaining the company from scratch every morning. While skills have some similar characteristics as a prompt, its quite a bit more than that.

The skill I run before every Cyborgs Writing session is called cyborgs-briefing. Its only job is to open my content plan, work out where I actually am, and hand me a short briefing before I start typing:

• current territory,

• how many posts are published,

• what’s next,

• what’s sitting half-drafted, and

• what’s still waiting in the ideas inbox.

Then it asks me one question to help me get started. I don’t need to load a project or cut and past context, or even work with a prompt.

For an ADHD mind that gets overwhelmed by over-planning, this helps me move forward today.

Subscribe now

Process and task aren’t the same thing

The skill below is organized by information type: concept, reference, principle, process, task.

These are the same labels I use across the whole knowledge base, on notes and skills alike. Most of those are self-explanatory. Two of them get confused constantly.

A process is an overview of the workflow, which provides context for the task. Without this, the tasks have less meaning.

A task is the specific instruction to run that procedure right now. Exact steps that the agent uses to accomplish the goal.

Collapsing the two makes it harder to revise and could also cause problems with the AI workflow.

➡️ Paid subscribers can see the structured prompt below ... but I’m also building a public, living version of every skill in this knowledge base, so the library updates as I revise instead of going stale.

For now, that library lives in a password-protected Obsidian vault. Paid subscribers can request access below.

Right now there’s 15 skills, but will be growing.

Read more

I’ve spent summers building Substack without a system, and it always breaks the same way: drafts accumulate, ideas scatter, deadlines slip when the semester hits. This year I built a visual map called The Churnivore, a project board for tracking status, and a Markdown archive to turn my scattered thinking into a legible, sustainable workflow where AI can actually participate.

This is the first summer in a long time when I am mostly free.

I’m not teaching.

I’m not buried in consulting.

I have research to work on, but I also have room to build things I’ve wanted for awhile:

• a local AI knowledge base,

• a personal RAG system,

• and a better content workflow.

And, of course, time to write.

That last part is where things can really get overwhelming for me.

I have no shortage of ideas for Cyborgs Writing. Too many ideas has usually been the problem. My ADHD brain is good at generating possible posts, series, experiments, frameworks, and rabbit holes. It is less good at turning all of that motion into a publishing plan that survives more than a few weeks.

So I tend to go with the flow.

That works for a while. Sometimes it works better than planning, because I’m consistently engaged and discovering new thoughts and ideas. So I don’t necessarily want to flatten that out entirely.

But a Substack eventually becomes something you have to manage, especially when the busyness of a semester comes barreling into my writing habits.

Drafts accumulate.

Notes scatter.

Themes recur.

Arguments get halfway built and then left open.

Readers also have a reasonable expectation that you will show up again next week.

And somewhere in the distance, next next year’s dragon is already coming down the tracks.

The Map

So this summer I’m using the open space to do something I usually avoid. Build a content plan.

I don’t really like planning in the abstract or ticking boxes. Administrative rituals too often remind me I have failed to perform or that I’ve missed something (or how much I still have to do).

Any system that depends on my dutiful attention to tiny status updates will eventually break.

So I drew a monster.

The monster is called The Churnivore. It sits at the end of a winding path, guarding a pile of gold that represents a modest but meaningful goal of building Cyborgs Writing into a sustainable publication over the next year.

The path has five regions. Every week between now and next July is a milestone. Each published post moves me a little farther along the trail.

It looks a bit ridiculous, but hopefully that’s why it’ll work.

A content calendar tells me what is supposed to happen. The map tells me where today sits in a larger arc. It turns the year into a quest instead of a spreadsheet. I can see which themes are moving, which ones are neglected, and how much of the path is still ahead.

And, ultimately, I’m hoping it gets me ahead of schedule, instead of writing “just-in-time.”

The Workflow Behind the Map

But the map can’t carry the whole workflow, or else it becomes another project-management chore.

The actual workflow lives in a project board. I use Blue to track whether a post is an idea, in progress, scheduled, or published. A draft ready for human editing gets a card. A scheduled post moves forward. A published post gets its link added back to the system, then archived in my Markdown knowledge base.

The map follows this board (connected to Claude via an MCP).

I’ve also created a complete Markdown archive of all my writing, where the publish versions are stored with all the appropriate metadata like dates, related ideas, and reusable components.

Each part protects a different kind of work. Production belongs on the board. The year planning belongs on the map. The reusable thinking is in the knowledge base.

AI can participate in that system without being asked to invent the whole thing from scratch.

That has become one of my main lessons from working with AI over the past year. The model is rarely the only variable that matters. The structure around the model often matters more.

A post with a status, pillar, date, and link becomes readable to the system. A published piece with a predictable home can be archived. Ideas tied to the map are easier to retrieve than ideas scattered across half my digital life.

The AI is not making the system smart.

The system is making the work legible.

The Five Regions

The map has five regions, each representing a theme I’ll explore this year:

• KB Building — Markdown knowledge bases, personal archives, and systems for making your own thinking retrievable.

• Info Types — Structured prompting, information architecture, and designing content that AI can actually use.

• AI Portfolios — How writers, students, and professionals can document and demonstrate their work with AI.

• World-Building — Collaborative fiction, interactive storytelling, and AI as a creative partner.

• Religion — Faith, meaning-making, and the deeper questions that emerge when machines start to think.

For now, I want to mark the shift.

I am not trying to publish more by sheer force of will.

I am trying to build a system that can hold more of the work than my attention can hold on its own.

The Churnivore is not really a monster outside the system. It is what happens when you don’t have a plan.

This year, I’m walking the path. The Churnivore is waiting. But for the first time, so is a map … that is, a structured plan.