← Back to Agent Instructions Drift Check
AGENTS.md, CLAUDE.md, .cursorrules, and
.github/copilot-instructions.md exist to tell an AI coding
agent (or a new human contributor) how a repo actually works: which files
to read first, which command runs the tests, which script builds the
project. That only stays true while every path, script name, and command
it names still matches what the repo actually has — and nothing about how
most repos are built keeps that true on its own.
A refactor renames scripts/check.py to
scripts/validate.py, or a package.json script
gets renamed from lint to lint:fix. The change
works, the tests pass, and the PR merges. Updating every instruction file
that named the old path or the old script is a manual step no build or
type check enforces — the same class of drift as an .env.example
falling out of sync with the code that reads it, just with a different
reader on the other end.
A human contributor who hits a broken command in a README usually
recognizes the failure for what it is and works around it. An agent
following the same instruction can spend an entire turn, or several,
treating a wrong command's output as a real signal about the code — the
sharpest version of this is the interpreter-mismatch footgun this tool
names directly: a bare pytest or mypy that
resolves to an isolated interpreter missing the repo's own dependencies
reports a plausible-looking "module not found," which reads exactly like a
real regression until someone notices which Python actually ran.
| Check | Source | Out of scope for v0 |
|---|---|---|
| Broken file reference | Markdown link targets and inline `code spans` that look like a path | Bare prose ("the config lives in src"), and fenced code blocks (too noisy — command flags and globs look path-shaped too) |
| Undefined npm script | npm run <name>, npm test, npm start | Makefile targets, Python tox/nox sessions, and any script invoked without the literal npm run form |
| Interpreter-mismatch footgun | Bare pytest / mypy inside a fenced code block with no python3 -m prefix on the same line | Any other tool with the same shim problem; only the two most commonly documented ones ship in v0 |
Run it on every pull request that touches your instruction file or the paths it names:
- name: Check AGENTS.md for drift run: node ventures/agent-instructions-drift-check/cli.mjs check AGENTS.md --root .
The command exits non-zero if any file reference is broken, any named npm script is undefined, or a known footgun is present — so the same pull request that renames a file or a script is the one that has to update the instruction file that named it.
A general markdown link checker (lychee, markdown-link-check) verifies that a link resolves to a file or URL that exists, which overlaps with this tool's file-reference check. It does not check npm-script references against package.json, and it does not know about the interpreter-mismatch shell footgun this tool looks for -- both specific to the failure modes an AI-coding-agent instruction file actually causes when it goes stale.
Because that is how instruction files actually cite a path when they mean it literally -- both this repo's own AGENTS.md and CLAUDE.md follow exactly that convention throughout. Scanning ordinary prose for anything path-shaped would flag version numbers, package names, and ordinary sentences far more often than it would catch a real broken reference.
No, and this is a named limitation rather than a hidden one. A sentence that names a full path once and then refers back to it later with just the filename ("the same validate.py as above") will report the later, shorter reference as not found even though the earlier, fuller path exists. The fix in the instructions file is usually to repeat the full path, which also makes the file easier for a human skimmer to follow.