@natjswenson/devlog 0.11.0 → 0.11.1

package/README.md

@@ -0,0 +1,272 @@
+# devlog
+
+[![npm](https://img.shields.io/npm/v/@natjswenson/devlog?color=blue)](https://www.npmjs.com/package/@natjswenson/devlog)
+[![license](https://img.shields.io/npm/l/@natjswenson/devlog)](./LICENSE)
+[![security](https://img.shields.io/badge/security-audited-green)](./SECURITY.md)
+[![vulnerabilities](https://img.shields.io/badge/npm%20audit-0%20issues-brightgreen)](#security)
+
+A Claude Code skill that turns each version release (a git tag) into a published how-to guide, written in your own voice — and a React example for displaying it on your site.
+
+> **Build in public, by release.** Tag a release like you always do. Run `/devlog`. Each new version shows up on your site as a polished, end-to-end implementation guide — in your voice, with cited sources and the gotchas you actually hit — not raw commit messages.
+
+## Live example
+
+The skill is in production at [natejswenson.com/devlog](https://natejswenson.com/devlog), publishing to [github.com/natejswenson/daily-dev-log](https://github.com/natejswenson/daily-dev-log). What you see on that page is exactly what `npx @natjswenson/devlog preview` renders for you locally.
+
+## How it works
+
+1. **You ship a release** — tag it (e.g. `git tag v0.3.0`), like you already do.
+2. **Run `/devlog` in Claude Code.** The skill runs `devlog scan` to find tags that don't yet have an entry, researches the engineering topic behind each release, and writes a full setup → build → use → verify how-to guide (with a required `## Gotchas` section mined from your real fix commits, and 3+ cited sources). Every draft passes a deterministic lint (`devlog lint-post`) plus a quality self-review before it publishes. It's idempotent — re-running does nothing until you cut a new release, and a published entry is never overwritten (`devlog publish-entry` refuses).
+3. **Your site fetches it.** Static `manifest.json` + per-release markdown files served from `raw.githubusercontent.com` — no backend needed.
+
+You can also manage configuration conversationally in Claude Code: "add this repo to devlog", "stop tracking X", "set min sources to 4", or `/devlog status` to see what would be generated without writing anything.
+
+**In your voice.** Entries are written using a voice profile, resolved in this order: your `config.voicePath` → [ghostwriter](../ghostwriter)'s `voice/` dir if installed → a bundled default. devlog reads `voice-profile.md` (and `voice-notes.md` overrides) — never ghostwriter's `algorithm.md`, since LinkedIn reach tuning doesn't apply to a dev log.
+
+## Quick start
+
+```sh
+npx @natjswenson/devlog init
+```
+
+That command:
+- Creates `<your-username>/daily-dev-log` on GitHub (or uses an existing one)
+- Installs the skill at `~/.claude/skills/devlog/`
+- Writes `~/.claude/skills/devlog/config.json` with your answers
+- Lets you register one or more projects in a single run
+
+Then:
+
+```sh
+npx @natjswenson/devlog preview
+```
+
+to see your dev log rendered locally at `http://localhost:5173`.
+
+## Prerequisites
+
+- **Node 18+** — for the CLI and preview app
+- **GitHub CLI** (`gh`), authenticated with `gh auth login` — used to create your dev-log repo and push entries
+- **Claude Code** — to run the `/devlog` skill
+
+## Commands
+
+| Command | What it does |
+|---|---|
+| `npx @natjswenson/devlog init` | One-time setup: create dev-log repo, install skill, write config |
+| `npx @natjswenson/devlog add-project` | Register a project (interactive; `--yes --path <p>` for non-interactive/agent use) |
+| `npx @natjswenson/devlog remove-project <key> --yes` | Unregister a project (published entries stay) |
+| `npx @natjswenson/devlog set <field> <value>` | Update one config field (`targetRepo`, `branch`, `gitAuthor`, `githubUser`, `voicePath`, `deepDive.minSources`, `deepDive.topicDomains`) |
+| `npx @natjswenson/devlog config [--json]` | Show your current config with validation status |
+| `npx @natjswenson/devlog scan [--project <key>]` | JSON plan of new releases needing entries (used by the skill) |
+| `npx @natjswenson/devlog lint-post <file>` | Deterministic post-contract check (used by the skill) |
+| `npx @natjswenson/devlog publish-entry ...` | Copy a drafted entry into a clone + update the manifest; never overwrites (used by the skill) |
+| `npx @natjswenson/devlog preview` | Run a local preview at `http://localhost:5173` |
+| `npx @natjswenson/devlog --help` | Usage |
+| `npx @natjswenson/devlog --version` | Version |
+
+> **Tip:** run from any directory *outside* a clone of this repo. Running inside the repo causes a `package.json` name collision and `npx` fails with `command not found`.
+
+## What you end up with
+
+```
+~/.claude/skills/devlog/
+├── SKILL.md          # The /devlog slash-command instructions
+├── config.json       # Your settings (mode 0600)
+└── voice/            # Bundled fallback voice profile (last resort)
+    ├── voice-profile.md
+    └── voice-notes.md
+
+github.com/<you>/daily-dev-log/   # Created by init, populated by /devlog
+├── myproject/
+│   ├── manifest.json
+│   ├── v0.3.0.md
+│   ├── v0.2.0.md
+│   └── ...
+└── ...
+```
+
+## Manual setup (if you prefer)
+
+```sh
+gh repo create <you>/daily-dev-log --public --add-readme
+mkdir -p ~/.claude/skills/devlog/voice
+curl -o ~/.claude/skills/devlog/SKILL.md https://raw.githubusercontent.com/natejswenson/devlog/main/SKILL.md
+# Optional fallback voice profile (used when voicePath and ghostwriter are both absent):
+curl -o ~/.claude/skills/devlog/voice/voice-profile.md https://raw.githubusercontent.com/natejswenson/devlog/main/voice/voice-profile.example.md
+curl -o ~/.claude/skills/devlog/voice/voice-notes.md https://raw.githubusercontent.com/natejswenson/devlog/main/voice/voice-notes.example.md
+# Then copy config.example.json → ~/.claude/skills/devlog/config.json and fill it in
+```
+
+## Add to your site
+
+### React (drop-in)
+
+```sh
+cp -r examples/react/ your-site/src/devlog/
+```
+
+Edit `your-site/src/devlog/devlog-config.js` to point at your repo, then:
+
+```jsx
+import DevLogPage from './devlog/DevLogPage.jsx';
+
+<DevLogPage project="myproject" />
+```
+
+Full instructions: [`examples/react/README.md`](./examples/react/README.md).
+
+### No site yet?
+
+The `preview/` directory is a complete deployable Vite app. Set `VITE_DEVLOG_OWNER` / `VITE_DEVLOG_REPO` / `VITE_DEVLOG_PROJECTS` env vars on Vercel, Netlify, or Cloudflare Pages, build with `vite build`, deploy `dist/`. See [`preview/README.md`](./preview/README.md).
+
+### Other stacks (Next, Astro, plain HTML)
+
+It's static JSON and Markdown on GitHub. Build whatever UI you want — see the **Data contract** below.
+
+## Data contract
+
+The dev-log repo has this layout, all served as raw files from `https://raw.githubusercontent.com/<owner>/<repo>/<branch>/`:
+
+```
+<repo>/
+└── <project-key>/
+    ├── manifest.json          # Index of all entries (newest first)
+    ├── v0.3.0.md              # One entry per release (named by version)
+    ├── v0.2.0.md
+    └── ...
+```
+
+**`manifest.json`:**
+```json
+{
+  "entries": [
+    { "date": "2026-06-08", "file": "v0.2.0.md", "title": "...", "summary": "...", "version": "v0.2.0" }
+  ]
+}
+```
+
+Strict validation rules (entries that don't match are silently dropped by the React example):
+- `date` matches `YYYY-MM-DD` (the release/tag date)
+- `file` matches `^[a-zA-Z0-9._-]+\.md$`
+- `title` and `summary` are non-empty strings
+- `version` (optional) matches `^[a-zA-Z0-9._-]+$`
+
+**Entry markdown:**
+
+```markdown
+---
+title: "Essay-style title (never 'release vX.Y.Z')"
+date: 2026-06-08
+project: myproject
+version: v0.2.0
+tags: [reliability, python]
+summary: "1-2 sentence hook framing the how-to"
+---
+
+## Shipped
+What the release delivered, then the pivot to the topic the guide teaches.
+
+## <Setup / build / use-and-verify sections>
+An end-to-end implementation guide a reader can follow without your repo:
+complete, language-tagged code blocks; reader-side verification steps.
+
+## Gotchas
+Real traps from the release's history, each: trap → symptom → escape.
+
+## Sources
+- [Title](https://...) — what it supports  (3+ distinct URLs)
+
+## Changelog
+- commit message ([abc1234](https://github.com/.../commit/abc1234567...))
+```
+
+That's the entire contract — `npx @natjswenson/devlog lint-post <file>` checks the mechanical parts of it.
+
+## Configuration reference
+
+`~/.claude/skills/devlog/config.json`:
+
+| Field | Type | Description |
+|---|---|---|
+| `targetRepo` | `"<owner>/<repo>"` | Repo where dev log entries are published. Must match `^[a-z0-9][a-z0-9._-]*\/[a-z0-9][a-z0-9._-]*$`. |
+| `branch` | string (optional) | Branch in the dev-log repo. Defaults to `main`. Must not contain `..` or start with `-`. |
+| `gitAuthor` | string | Your name. Retained for backward compatibility; it is **not** currently rendered on entries (the author filter was removed, and release notes summarize all commits in a tag range). Still **required** by config validation — it must be present and non-empty (don't drop it). Whitespace OK; no shell metacharacters. |
+| `githubUser` | string | Your GitHub username. |
+| `voicePath` | string (optional) | Directory holding `voice-profile.md` (and optionally `voice-notes.md`) used to write entries in your voice. A leading `~` is expanded. If unset, devlog uses ghostwriter's `voice/` if installed, else the bundled default. Read only — never shell-interpolated. |
+| `projects` | array | One entry per project you want dev logs for. |
+| `projects[].key` | string | Subdirectory name in the dev-log repo. Strict token: `^[a-z0-9][a-z0-9._-]*$`, no `..`. |
+| `projects[].label` | string (optional) | Display name for the tab. Defaults to `key`. |
+| `projects[].path` | string | Local filesystem path to the project. Whitespace OK. |
+| `projects[].remote` | `"<owner>/<repo>"` | The project's GitHub remote. Used to mark public commits and link them. |
+| `projects[].pathFilter` | string (optional) | Repo-relative subdir scoping this project's commits in a monorepo (e.g. `skills/devlog`). |
+| `projects[].tagPrefix` | string (optional) | Prefix of the git tags that mark this project's releases (e.g. `devlog-v`). Defaults to `v`. Used in `git tag --list '<tagPrefix>*'`. |
+| `deepDive` | object (optional) | Controls the researched how-to. `topicDomains` (array, default `["AI", "DevOps/SRE", "software engineering"]`) steers topic selection; `minSources` (integer 1-10, default 3) is the floor of **distinct** cited source URLs per post. |
+
+See [`config.example.json`](./config.example.json) for a complete template, or run `npx @natjswenson/devlog config` to inspect your current config with validation.
+
+## Security
+
+The package is designed to be safe to install on a developer's machine and have ambient gh/git credentials. See [SECURITY.md](./SECURITY.md) for the full threat model and audit history.
+
+**At a glance:**
+- ✓ All shell calls in the CLI use `spawnSync` with argv arrays (no shell, no injection surface)
+- ✓ The skill validates every config field against an allowlist before interpolating into shell commands; instructs the LLM to single-quote interpolated values
+- ✓ Markdown rendering uses `react-markdown` with `skipHtml` and an allowlist `urlTransform` (only http(s)/mailto allowed; data:, javascript:, vbscript:, file:, blob: all neutralized)
+- ✓ Frontmatter parser uses `Object.create(null)` + key allowlist (no prototype pollution)
+- ✓ All external JSON is schema-validated before use (manifest, env-var projects array)
+- ✓ Vite dev server bound to `localhost`, CORS off
+- ✓ Demo-mode `window.fetch` override gated to dev builds only
+- ✓ `config.json` written atomically (tmp + rename), mode 0600
+- ✓ All dependencies pinned to exact versions
+- ✓ `npm audit`: 0 known vulnerabilities
+
+**To report a vulnerability:** open a [GitHub security advisory](https://github.com/natejswenson/devlog/security/advisories/new). Do not open a public issue.
+
+## Customization
+
+- **Tweak the entry template:** edit `~/.claude/skills/devlog/SKILL.md` (Generate mode, Step 3 — the how-to contract).
+- **Tweak your voice:** edit the `voice-profile.md` / `voice-notes.md` in your `voicePath` (or `~/.claude/skills/devlog/voice/`).
+- **Tune research depth:** `npx @natjswenson/devlog set deepDive.minSources 4`, or `set deepDive.topicDomains "security, platform engineering"`.
+- **Tweak the UI:** override the `--devlog-*` CSS variables in `examples/react/DevLogPage.css` to match your theme.
+- **Add or remove projects:** `npx @natjswenson/devlog add-project` / `remove-project <key> --yes` — or just tell Claude ("add this repo to devlog").
+
+## Testing & evals
+
+The deterministic core (`lib/`, the CLI) is covered by `npm test` (node:test; scan tests
+run against real throwaway git repos). The non-deterministic half — post quality — is
+covered by a cost-capped eval harness:
+
+```sh
+node evals/run_eval.mjs --mock    # $0, runs in CI; deterministic layer only
+node evals/run_eval.mjs --live    # LLM judge on golden fixtures; quotes spend, hard cap $0.50
+```
+
+The judge scores reproducibility, code completeness, gotcha quality, citations, voice,
+and scope honesty (pass ≥ 7/10). Without `ANTHROPIC_API_KEY` in the environment, mock
+mode is forced — CI can never spend money.
+
+## Troubleshooting
+
+**`sh: devlog: command not found` when running `npx`:** you're inside a checkout of this repo. The local `package.json` name collides with the published one. Run `npx` from somewhere else, e.g. `cd ~ && npx @natjswenson/devlog ...`.
+
+**Init prompts show `78` after placeholder text:** that's an artifact of how some output capture tools render `\x1b7`/`\x1b8` (cursor save/restore) escape sequences. In a real interactive terminal, you won't see it.
+
+**Preview is blank / no tabs / no entries:** check the browser console. If you see CJS interop errors related to `react-dom/client`, `style-to-js`, or `react-markdown`, you're on a pre-0.1.6 release — upgrade with `npm cache clean --force && rm -rf ~/.npm/_npx && npx --yes @natjswenson/devlog@latest preview`.
+
+**Init can't find `gh`:** install via [cli.github.com](https://cli.github.com/) and run `gh auth login`.
+
+**`Config validation failed: ...`** — the validator rejected something in your `config.json`. Run `npx @natjswenson/devlog config` for a detailed diagnosis. Common causes:
+- A field has shell metacharacters (`;` `&` `|` `` ` `` `$` etc) — see SKILL.md for the full list
+- A project key contains `..` or `/`
+- A path doesn't exist on disk
+
+**Preview shows "Setup required":** you deployed the preview app standalone but didn't set the env vars. Set `VITE_DEVLOG_OWNER`, `VITE_DEVLOG_REPO`, `VITE_DEVLOG_PROJECTS` (JSON-stringified array) in your hosting environment.
+
+## Versioning
+
+Releases are documented in [CHANGELOG.md](./CHANGELOG.md). The package follows semver — bug fixes/security patches in patch releases (0.1.x), behavior changes in minor (0.x.0).
+
+## License
+
+MIT

package/evals/baseline/published/devlog-v0.10.0.md

@@ -0,0 +1,129 @@
+---
+title: "Folding a content repo into the site repo without breaking idempotent publishing"
+date: 2026-07-19
+project: devlog
+version: v0.10.0
+tags: [github-api, gh-cli, idempotency, cloudflare-pages, static-sites, content-pipeline, release-engineering, shell]
+summary: "Moving generated content into a subfolder of the site repo turns every publish push into a deploy. The catch: your publisher's already-published check is full of path assumptions, and missing one silently re-plans your entire archive."
+---
+
+## Shipped
+
+devlog v0.10.0 adds a `targetDir` setting: the publisher can now write its content tree into a subdirectory of the target repo instead of the repo root. That's the piece that let me delete a whole repo from my pipeline; devlog entries now land in `content/devlog/` inside the site repo itself, and the push that publishes them is the same push that deploys the site. The release also rewrote the cover style guide for the site's PRESS brand (including a fill floor for the hero illustration) and froze entry numbers at publish time so a backdated entry can't renumber the archive. This post is about the migration pattern: what an idempotent publisher checks before it writes, and every place a path prefix has to reach when the content moves.
+
+## The two-repo tax
+
+The old shape was a dedicated content repo. The generator pushed markdown, a manifest, and a cover PNG there; the site fetched it all at build time. Workable, but every publish needed a second step, an empty "rebuild" commit to the site repo, because [Cloudflare Pages triggers a deployment on commits to the production branch](https://developers.cloudflare.com/pages/configuration/branch-build-controls/) of the repo it watches, and content landing in a *different* repo doesn't count.
+
+Move the content into the site repo and that second step disappears. The site reads the files from disk at build time, and the publish push is the deploy trigger. The only real engineering is in the publisher: it has to stay idempotent when its target is no longer a repo root.
+
+## The check that makes publishing idempotent
+
+A release publisher should be safe to re-run. Mine plans work by listing what's already published and diffing against local git tags, one directory listing per project via the [GitHub contents API](https://docs.github.com/en/rest/repos/contents): `GET /repos/{owner}/{repo}/contents/{path}` returns an array of entries when `path` is a directory, and `ref` pins the branch. Through [`gh api`](https://cli.github.com/manual/gh_api) that's a one-liner, authenticated with your existing CLI login:
+
+```bash
+gh api "repos/OWNER/REPO/contents/PROJECT?ref=main" --jq '.[].name'
+```
+
+The Node version, with the two failure modes that matter kept distinct:
+
+```js
+// existing.mjs
+import { spawnSync } from 'node:child_process';
+
+// Which entry files already exist for one project.
+// Returns { files: Set, status: 'ok' | 'empty' | 'failed' }.
+export function fetchExistingEntries(repo, branch, projectKey, targetDir = '') {
+  const contentPath = targetDir ? `${targetDir}/${projectKey}` : projectKey;
+  const r = spawnSync('gh', ['api', `repos/${repo}/contents/${contentPath}?ref=${branch}`, '--jq', '.[].name'], { encoding: 'utf8' });
+  if (r.status === 0) {
+    return { files: new Set(r.stdout.split('\n').filter(Boolean)), status: 'ok' };
+  }
+  // 404 means "this project has no entries yet"; a normal state for a new
+  // project, not an error. Anything else means the check itself failed and
+  // the caller should know its already-published filter may be incomplete.
+  if (/HTTP 404|Not Found/i.test(r.stderr)) {
+    return { files: new Set(), status: 'empty' };
+  }
+  return { files: new Set(), status: 'failed' };
+}
+```
+
+The `targetDir` parameter is the whole feature. Without it, moving content to `content/devlog/` means the check asks GitHub for `contents/ghostwriter` at the repo root, gets a 404, concludes "no entries yet", and the planner happily re-plans every release you've ever published. Fifty-five posts, in my case, all queued for regeneration against a publisher that would then refuse each one.
+
+## Thread the prefix everywhere, and validate it
+
+A path that ends up inside shell commands and API URLs earns strict validation at config time:
+
+```js
+// config.mjs
+export function validateTargetDir(targetDir) {
+  if (
+    typeof targetDir !== 'string' ||
+    !/^[A-Za-z0-9._-]+(\/[A-Za-z0-9._-]+)*$/.test(targetDir) ||
+    targetDir.split('/').some((s) => s === '.' || s === '..')
+  ) {
+    throw new Error(`targetDir must be a relative path like "content/devlog": got ${JSON.stringify(targetDir)}`);
+  }
+  return targetDir;
+}
+```
+
+Relative only, tight charset, no traversal segments. The regex alone would accept `..` as a "word", so the explicit segment check backs it up.
+
+Then find every consumer of the old repo-root assumption. In my publisher there were three:
+
+```js
+// scan.mjs: the planner threads targetDir into the existence check and
+// echoes it in its output, so the publish step can build paths from the
+// same value instead of re-reading config.
+const existing = fetchExistingEntries(config.targetRepo, branch, project.key, config.targetDir || '');
+```
+
+The other two live in the publish step. The writer's content root becomes `<clone>/<targetDir>` while git still operates on the clone root; and any "view it here" URL you print needs the prefix too:
+
+```bash
+git clone --depth=1 "https://github.com/OWNER/REPO.git" "$TMP/REPO"
+CONTENT_ROOT="$TMP/REPO/content/devlog"   # writers target this
+# ... write PROJECT/vX.Y.Z.md, PROJECT/manifest.json, PROJECT/vX.Y.Z.png under $CONTENT_ROOT ...
+git -C "$TMP/REPO" add . && git -C "$TMP/REPO" commit -m 'devlog: add release entries'
+git -C "$TMP/REPO" push origin main       # on Cloudflare Pages, this IS the deploy
+```
+
+## Cut over and verify
+
+The safe order matters: land the content move in the site repo first, deploy it, and only then point the publisher at the new target; the existence check reads the target's live branch, so flipping config before the content exists there recreates the 404-means-empty re-planning problem on purpose.
+
+After flipping, one scan tells you whether the prefix reached everywhere. This is the real output from my cutover:
+
+```text
+targetRepo: natejswenson/natejswenson.io | targetDir: content/devlog | branch: main
+totalNewReleases: 5
+  local-fitness: existenceCheck=ok new=0
+  devlog: existenceCheck=ok new=2
+  ghostwriter: existenceCheck=ok new=2
+  resume: existenceCheck=ok new=0
+  local-budget: existenceCheck=ok new=0
+  personal: existenceCheck=ok new=0
+```
+
+Every project resolves `ok` through the subfolder, and the only "new" releases are genuinely untagged ones. If you see a fully published project reporting `new=<its entire history>`, the prefix missed the existence check.
+
+## Gotchas
+
+- **The missed-prefix failure is silent and looks like work to do.** Trap: any consumer of the old root-relative path that you didn't update. Symptom: not an error; the scan cheerfully reports your whole archive as new releases. Escape: after any path change, run the planner against the live target and assert the re-plan count is zero before letting anything publish.
+- **404 is a state, not a failure.** Trap: treating every non-200 from the contents API the same. Symptom: either a brand-new project blocks publishing (404 treated as failure) or a real outage quietly re-plans everything (failure treated as empty). Escape: three-way status, as in `fetchExistingEntries` above; only 404 means empty.
+- **The check is a filter, not the safety.** Trap: trusting the remote listing as the last line of defense. Symptom: a degraded check plus an overwrite-happy writer equals clobbered history. Escape: the writer itself must refuse to overwrite an existing entry against the fresh clone; then a failed existence check degrades to wasted planning, never to data loss.
+- **zsh eats the `?` in the API path.** Trap: pasting `gh api repos/o/r/contents/x?ref=main` unquoted into zsh while debugging. Symptom: `no matches found: repos/...` before gh even runs, because `?` is a glob character. Escape: quote the whole endpoint argument; I hit this within an hour of shipping the feature.
+- **Directory listings cap at 1,000 files.** Trap: one flat directory per project, forever. Symptom: the [contents API stops listing past 1,000 entries](https://docs.github.com/en/rest/repos/contents) and the existence filter goes blind. Escape: at that scale, switch the check to the Git Trees API; per-project directories buy a lot of headroom first.
+
+## Sources
+
+- [GitHub REST API: repository contents](https://docs.github.com/en/rest/repos/contents) — directory responses are arrays, `ref` pins the branch, 404 for missing paths, 1,000-file listing cap
+- [Cloudflare Pages: branch build controls](https://developers.cloudflare.com/pages/configuration/branch-build-controls/) — deployments trigger on commits to the production branch
+- [gh api manual](https://cli.github.com/manual/gh_api) — authenticated API calls from the CLI, `--jq` for response filtering
+
+## Changelog
+
+- feat(devlog): targetDir — publish the content tree into a subdirectory of targetRepo (#84) ([5bfada2](https://github.com/natejswenson/claude-skills/commit/5bfada2bfd3d0a3826cb78c1eed0056d71436840))
+- feat(devlog): PRESS cover style guide + frozen entry numbers ([2627a4c](https://github.com/natejswenson/claude-skills/commit/2627a4ce8cd59dfd95010e49f0a274c23abce289))

package/evals/baseline/published/devlog-v0.11.0.md

@@ -0,0 +1,139 @@
+---
+title: "Auditing an AI skill against its own past runs"
+date: 2026-07-19
+project: devlog
+version: v0.11.0
+tags: [ai-agents, agent-evaluation, postmortems, claude-code, subagents, git-forensics, guardrails, transcripts]
+summary: "Version 0.11.0 of this skill came entirely out of one exercise: asking the model to review the skill's own previous six runs. Here is the four-lens audit loop that produced it, and what the round cost in tokens."
+---
+
+## Shipped
+
+devlog 0.11.0 is a batch of guardrails: tombstones so a deleted entry can never republish itself, a ground-truth gate that verifies every claim about my own repos against git before publishing, a mechanical version of the run-the-code check, deterministic voice linting, and a fix for a CLI command that silently ignored fresh input. None of it came from a feature idea. All of it came from asking the model to review the skill's own past six runs. That review loop is the technique worth teaching; this guide walks through running it on your own agent skill.
+
+## The four lenses
+
+An agent skill drifts quietly. No single run fails hard enough to file a bug, but small problems repeat until they are load-bearing. The fix is the same discipline SRE applies to incidents: document what happened, understand the causes, and put preventive actions in place so it stops recurring ([Google SRE book, postmortem culture](https://sre.google/sre-book/postmortem-culture/)). The difference is that your "incident" is spread across runs, so you have to go collect it.
+
+I review on four dimensions, each answering one question:
+
+- **Accuracy**: did the runs do what the skill promises? For a writing skill, are the claims in the output true?
+- **Completeness**: what did the runs miss, and what did a human have to clean up afterward?
+- **Efficiency**: where did tokens and wall-clock time go that produced no value?
+- **Agent UX**: where did the skill's own instructions or tools fight the agent executing them?
+
+The last one surprises people. The agent is a user of your skill, and it hits usability bugs the same way humans do; it just cannot file a complaint. You find those bugs in transcripts.
+
+## Three evidence streams
+
+A run leaves three artifact trails, and each lens needs a different one.
+
+First, the outputs themselves: whatever the skill publishes. Grade a sample against the skill's own quality contract, and verify factual claims against the system of record instead of taking the output's word for it.
+
+Second, the correction commits. If your skill writes into a git repo, every manual fix a human made after a run is a finding with a timestamp. Separate the skill's own commits from everything else:
+
+```bash
+# Commits the skill's happy path writes (use your skill's commit message):
+git log --date=short --pretty='%h %ad %s' --grep='add release entries'
+
+# Everything a human had to do around them:
+git log --date=short --pretty='%h %ad %s' --invert-grep --grep='add release entries'
+```
+
+Running that against my dev-log repo (output trimmed to the signal lines):
+
+```text
+9b26494 2026-07-18 devlog: add release entries
+8f8938a 2026-07-17 devlog: add release entries
+...
+83cd9de 2026-07-18 chore: assign permanent entry numbers to all 55 entries
+c160db1 2026-07-17 devlog: file market-research v0.1.0 entry under personal
+```
+
+The second list held the review's best material: an entry reverted the same day a run re-added it, three posts consolidated into one by hand, and a moved file (that `c160db1` line) that the next run would have silently regenerated. Manual cleanup is the skill telling you what it cannot do yet.
+
+Third, the transcripts. Claude Code stores session logs as JSONL under `~/.claude/projects/`, one directory per working directory. Find the sessions where your skill ran by grepping for a command only it uses:
+
+```bash
+grep -l 'devlog scan --json' ~/.claude/projects/*/*.jsonl
+```
+
+Then pull rough per-run metrics. Transcript size is a fair cost proxy, and counting CLI invocations or retries shows where the loop stalled:
+
+```bash
+f=$(grep -l 'devlog scan --json' ~/.claude/projects/*/*.jsonl | head -1)
+wc -c < "$f"                      # transcript bytes, a rough cost proxy
+grep -c '"name":"Bash"' "$f"      # shell tool calls in the session
+```
+
+```text
+ 2762710
+95
+```
+
+## Fan out reviewers that can touch ground truth
+
+One reviewer reading everything runs out of context and blends the lenses together. I dispatch one subagent per evidence stream and give each a narrow brief plus access to the real repos. The prompt shape matters more than the wording:
+
+```markdown
+You are auditing the output quality of the <skill> agent skill.
+The skill's contract is at <path to SKILL.md>; read it first.
+
+For each published output:
+1. Grade it against the contract, point by point.
+2. Verify its factual claims against the SOURCE repo's git history
+   (tags, diffs, commit messages), not against the output's own text.
+3. Report patterns across outputs, each with concrete evidence
+   (file, commit hash, or quote).
+
+Be skeptical and specific; this feeds a fix list, not a report card.
+```
+
+Point 2 is the one you cannot skip. My accuracy reviewer found a published post whose central premise was false: it claimed two of four packages in a release never got git tags, and a single `git tag -l` showed all four tags existed. The post had passed the skill's self-review, because self-review graded the draft against the writing contract, never against the repo. A reviewer without repo access would have graded the same lie the same way.
+
+## Ship findings as gates, then re-run
+
+A review that produces a document has not improved anything yet. The SRE postmortem bar applies: findings become prioritized action items or they are theater ([postmortem culture](https://sre.google/sre-book/postmortem-culture/)). For agent skills I hold a stricter line: every finding ships as a mechanical gate in the next version, because an instruction the agent is supposed to remember is exactly the thing the review just proved gets skipped. Reflexion showed that agents improve when reflections persist somewhere durable instead of evaporating with the episode ([Shinn et al., 2023](https://arxiv.org/abs/2303.11366)); for a skill, the durable place is the code path the agent cannot route around.
+
+Two examples from this round. The false-premise finding became a required pre-publish step: list every claim the draft makes about your own repo, then verify each with a git command run in that session, and delete what you cannot verify. The cleanup-commit findings became identity guardrails; a retired artifact now keeps a tombstone row in its manifest, and the publish path refuses it:
+
+```json
+{ "version": "v0.1.0", "file": "v0.1.0.md", "removed": true,
+  "reason": "consolidated into the 2026-07-17 entry" }
+```
+
+```js
+function refuseTombstoned(manifest, version) {
+  const tombstoned = manifest.entries.find(
+    (e) => e.removed && e.version === version,
+  );
+  if (tombstoned) {
+    throw new Error(
+      `${version} was editorially retired (${tombstoned.reason}); refusing to republish.`,
+    );
+  }
+}
+```
+
+Transcript findings usually fix the tool, not the prompt. Anthropic's agent guidance says to invest in the agent-computer interface the way you would invest in UI design ([Building effective agents](https://www.anthropic.com/engineering/building-effective-agents)), and my transcripts proved why: a render command that silently no-opped when its output already existed cost the agent a multi-call debugging dance in two separate runs. The fix was one reordered check in the CLI, worth more than any added instruction.
+
+Then re-run the skill. The next real run is the acceptance test: this post was generated by the version the audit produced, its opening scan reported the new CLI version, and the retired entry scanned as tombstoned instead of resurfacing as a new release.
+
+On cost, since I had never tallied it: the three review subagents in this round reported 337k tokens between them, and the three code-exploration agents that turned findings into an implementation plan reported another 273k. Call the whole round about 600k subagent tokens plus the main session. That is real money, and it bought eighteen files of shipped fixes; I would not spend it weekly, but per release milestone it has paid for itself every time.
+
+## Gotchas
+
+- **Self-review grades the essay, not the facts.** Trap: letting the skill's quality check compare the draft to a rubric while every factual claim goes unchecked. Symptom: a confident, well-structured output with a false premise sails through. Escape: give reviewers (and the skill itself) repo access and require a verification command per claim; a claim you cannot verify gets removed, not softened.
+- **Honor-system steps are skipped exactly when they matter.** Trap: writing "run the code blocks and check the output" as an instruction. Symptom: outputs that say "this is the real output" over code that cannot run; my audit found one whose demo files were never defined anywhere. Escape: turn the instruction into a command the agent runs (mine extracts a draft's code blocks into numbered files), so skipping it becomes visible instead of silent.
+- **Output-only reviews miss tool friction entirely.** Trap: auditing what the skill produced and never how the agent got there. Symptom: the outputs look fine while every run quietly burns calls fighting the same CLI quirk. Escape: make transcripts a first-class evidence stream; the silent no-op I fixed in 0.11.0 appeared in zero outputs and two transcripts.
+- **The review has a real price and nobody is tracking it.** Trap: running fan-out reviews on a schedule without measuring them. Symptom: a surprise in the usage dashboard. Escape: pull the token counts from your platform's subagent usage reports while the round is fresh, and set the cadence from the number instead of a hunch.
+
+## Sources
+
+- [Google SRE Book: Postmortem Culture](https://sre.google/sre-book/postmortem-culture/) — blameless postmortems, and findings becoming prioritized preventive action
+- [Anthropic: Building effective agents](https://www.anthropic.com/engineering/building-effective-agents) — measure and iterate; invest in the agent-computer interface like UI design
+- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://arxiv.org/abs/2303.11366) — agents improve when reflections persist in durable memory across episodes
+
+## Changelog
+
+- feat(devlog): 0.11.0 — tombstones, ground-truth gate, and the six-run audit fixes (#86) ([bd8fa5d](https://github.com/natejswenson/claude-skills/commit/bd8fa5d88a8c33dbf6f13ed64394b0682b4b8f4b))