Why do Markdown knowledge bases for AI agents go stale?

Short answer: a Markdown knowledge base rarely breaks with an error. It drifts quietly:

• pages that still exist get deleted;
• pages that were deleted stay;
• renamed pages lose their history;
• the sync says "healthy" while a source has stopped;
• the "last synced" time is wrong;
• an error message gets saved as if it were content;
• an update is skipped without a word;
• the source tool changes its format.

Your agent then answers from old files, with a confident citation.

I run a small team of AI agents: product, development, design and marketing. They take their tasks from Jira and write their plans and decisions as dated Markdown files. That works well, until the files and reality stop matching.

The first time it happened to me, it was my own fault. I made a decision in a conversation and never wrote it down. The agent kept working from the old decision. Nothing failed and nothing warned me. I only caught it because I happened to read the agent's plan.

That made me curious about how often this happens when the files are written by a sync, not by hand. So for a few weeks I tracked the public bug reports of tools that keep Markdown knowledge bases in step with other systems. The tools are good, and their maintainers fix things fast. But the same problems kept coming back. Here they are.

What does "stale" mean for an agent's knowledge base?

A file is stale when it no longer matches its source, and nothing tells you. The wrong content is only half of it. The other half is a signal that lies: a green health check, a fresh-looking "last synced" time, a sync that exits successfully.

In which ways does a Markdown knowledge base go stale?

These are the 8 failure modes I found in public reports from the last month:

1. Pages that still exist get deleted. One sync treated every file it couldn't see (for example, an ignored or uncommitted file) as deleted. It recorded about 19,000 deletions. A clean-up job later removed hundreds of pages for good, and they had to be restored from a backup.
2. Deleted pages don't go away. In other reports, approved deletions stayed "on hold", or old soft-deleted entries blocked every sync after them. People building their own search indexes say the same: deleted pages stay in the index.
3. Renamed pages lose their identity. When a page is tracked by its path, a rename looks like a delete plus a new page. One migration that guessed renames by similar content swapped the identities of 15 pairs of near-identical pages.
4. "Healthy" while a source has stopped. One account's access token was revoked. The health check still said everything was fine, and that source stopped updating for about 21 hours before anyone noticed. The sync imported nothing and reported success.
5. The right content with the wrong "last synced" time. The import worked, but the source still showed a sync date 11 days old. The freshness check skipped it, so its green result proved nothing.
6. An error message saved as knowledge. A connection failed with an authorization error. The tool recorded the pull as a success and wrote the error text into the files as content. The agent then learned it.
7. An update that checks nothing. An incremental update needed an earlier version to compare against. It was missing (a shallow checkout in CI, or a force-push), so the update moved forward without checking anything. Pages went stale silently.
8. The source tool changes its format. After a rendering change in a popular wiki, diagrams started to export as placeholder text. Some content types can't be exported through the API at all.

Why is this worse for an AI agent than for a person?

Because an agent doesn't get suspicious.

• A person who reads an old page often notices that something is off.
• An agent quotes it, cites it, and builds on it.

The citation makes the answer look more trustworthy, not less. In my case the agent did exactly what its files said. The files were just wrong.

How can you tell if your knowledge base has gone stale?

Check these, per source rather than overall:

• When was each source last checked successfully? Not "last run": last successful check.
• Did any source return nothing, or an error, on its last run? An empty result from a source that has content is a failure, not "no changes".
• Do the deletion counts make sense? A sudden jump in deletions usually means the sync lost access, not that people deleted pages.
• Are there pages with no source left? And sources with no pages?
• Pick 3 recently changed items in the source. Is the new version in the files?

What should a sync do to prevent it?

The fixes in those reports point to the same 7 rules:

1. Track each page by the source's own ID, not by its path or content. A rename then stays a rename.
2. Treat "missing" as "on hold", not "deleted". Delete only what the source says is gone.
3. Stop and ask before large deletions, and show the list.
4. Report status per source, and truthfully. "Last synced" must mean the content was actually checked.
5. Never write an error as content. A failed fetch writes nothing.
6. Run a full comparison now and then, because incremental shortcuts drift.
7. Say what couldn't be exported, and why, instead of leaving a silent gap.

How does Markdown Pipeline handle this?

I built Markdown Pipeline because I wanted my agents' files to come from the team's tools, and I wanted to trust them. Today it covers Jira Cloud and local folders, on Linux, in early access. It follows these rules, and I expect to find my own bugs too:

• Sources are only read. It never changes your Jira data or your folders.
• Deletions are picked up on the hourly background check.
• Big changes stop and ask. If a sync would delete or overwrite more than 40% of a pipeline's files (and at least 5), it writes nothing and shows you the list to approve.
• A missing or unreadable source folder is reported as an error, and nothing it owns is deleted.
• Your edits are kept. A file you edited by hand is skipped and reported, never silently replaced.
• mdpipeline status shows the last run, conflicts, the guard state and the age of your files.

Install it and start with the built-in sample data. No sign-in needed.

Have you seen a failure mode I missed? Tell me, and I'll add it.