@csark0812/skeleton 1.5.3 → 1.5.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +55 -29
- package/dist/cli.js +10 -5
- package/package.json +12 -1
package/README.md
CHANGED
|
@@ -2,15 +2,13 @@
|
|
|
2
2
|
|
|
3
3
|
**Source of truth for** Package overview.
|
|
4
4
|
|
|
5
|
-
<!-- doc-meta: owner=eng | last-reviewed=2026-07-
|
|
5
|
+
<!-- doc-meta: owner=eng | last-reviewed=2026-07-17 -->
|
|
6
6
|
|
|
7
7
|
Agent repos get messy fast. Skills get copied around, docs disagree, links go stale, and nobody remembers which file is actually canonical.
|
|
8
8
|
|
|
9
|
-
Skeleton is an SSOT linter for that layer.
|
|
9
|
+
Skeleton is an SSOT linter for that layer. Define the contract once; Skeleton checks it locally and in CI. If a canonical doc disappears, a registry drifts, a skill index stops matching disk, or a generated reference gets edited by hand, the audit fails before merge.
|
|
10
10
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
Think ESLint, but for the docs and skills your agents rely on.
|
|
11
|
+
Think ESLint — for the docs and skills your agents rely on.
|
|
14
12
|
|
|
15
13
|
Skeleton is **not** a runtime agent harness. It doesn't execute tools, enforce permissions, or manage memory. It checks whether the repo around those systems still holds together.
|
|
16
14
|
|
|
@@ -18,9 +16,7 @@ Skeleton is **not** a runtime agent harness. It doesn't execute tools, enforce p
|
|
|
18
16
|
|
|
19
17
|
Agents can read the repo. They can't reliably infer which of three conflicting docs wins, whether a synced skill should be edited here, or which validation command actually proves a change.
|
|
20
18
|
|
|
21
|
-
That
|
|
22
|
-
|
|
23
|
-
Skeleton turns those repo conventions into checks:
|
|
19
|
+
That needs to be explicit — and stay true after the next 50 PRs. Skeleton turns those conventions into checks:
|
|
24
20
|
|
|
25
21
|
| Code repos | Agent repos |
|
|
26
22
|
| ------------------------------------------------------- | -------------------------------------------------------------------------------------- |
|
|
@@ -28,20 +24,50 @@ Skeleton turns those repo conventions into checks:
|
|
|
28
24
|
| `eslint --fix` on changed files | `skeleton validate changed` on changed docs and skills |
|
|
29
25
|
| Pre-commit + CI gate | `--staged` pre-commit + `--base` CI gate |
|
|
30
26
|
|
|
31
|
-
Skill linters
|
|
27
|
+
Skill linters ask: _"Is this SKILL.md well-formed?"_
|
|
28
|
+
|
|
29
|
+
Skeleton asks the repo-level question: _"Does this whole thing still agree with itself?"_
|
|
30
|
+
|
|
31
|
+
## Why a clean SSOT helps agents
|
|
32
|
+
|
|
33
|
+
### Question
|
|
34
|
+
|
|
35
|
+
Does an intact Skeleton contract change agent behavior — grounding on the right doc, picking the right validation lane, and how much work it takes to get there?
|
|
36
|
+
|
|
37
|
+
### What we did
|
|
38
|
+
|
|
39
|
+
We ran a paired live A/B harness with [`@post-print/agent-test`](https://www.npmjs.com/package/@post-print/agent-test): `skeleton-clean` vs `skeleton-messy`. Same prompts and scenario set; the only intentional difference was registry / conflict structure and context profile.
|
|
40
|
+
|
|
41
|
+
Scenarios covered contested grounding (conflicting docs), docs-only validation routing, canonical grounding, owned-skill routing, and customize ownership. Protocol: **N=10** sequential paired compares on 2026-07-17; McNemar on paired pass/fail; median token deltas with a bootstrap CI on the mean.
|
|
32
42
|
|
|
33
|
-
|
|
43
|
+
Full method: [refs/llm-harness.md](refs/llm-harness.md). Suites: [agent-suites/README.md](agent-suites/README.md). Aggregated numbers: [SUMMARY.md](agent-suites/evidence/SUMMARY.md). Side-by-side excerpts: [evidence/transcripts/](agent-suites/evidence/transcripts/).
|
|
34
44
|
|
|
35
|
-
|
|
45
|
+
### What improved
|
|
36
46
|
|
|
37
|
-
|
|
47
|
+
|
|
38
48
|
|
|
39
|
-
|
|
40
|
-
- A [2025 METR randomized trial](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/) found experienced open-source developers took 19% longer with early-2025 AI tools while believing they were faster.
|
|
49
|
+
|
|
41
50
|
|
|
42
|
-
|
|
51
|
+
On tasks that depend on an intact SSOT, the clean fixture was both more accurate and cheaper:
|
|
43
52
|
|
|
44
|
-
|
|
53
|
+
- **Contested grounding** — In every paired run, clean settled on the registry canonical; messy never did (McNemar p = 0.002). Clean hops the registry; messy thrashes across conflicting docs.
|
|
54
|
+
- **Docs routing** — Clean consistently chose the correct audit lane; messy invented a non-existent `audit all` path (McNemar p = 0.002).
|
|
55
|
+
- **Token cost** — Across grounding tasks, messy used a median **~312k** more tokens than clean (bootstrap 95% CI on the mean excludes 0). Pass rate alone understates the gap: messy can still luck into an answer while spending far more.
|
|
56
|
+
|
|
57
|
+
Two scenarios did **not** show a clean accuracy win:
|
|
58
|
+
|
|
59
|
+
- **Canonical grounding** — Pass rates were close; the difference was not significant. Cost still favored clean.
|
|
60
|
+
- **Skill routing + customize** — Tied. The caller `AGENTS.md` already encodes both rules, so the fixtures did not separate on those prompts.
|
|
61
|
+
|
|
62
|
+
Charts regenerate from `SUMMARY.json` via `bun run agent:evidence:charts`.
|
|
63
|
+
|
|
64
|
+
### Limits
|
|
65
|
+
|
|
66
|
+
Fixture A/B on Skeleton’s own contract (live model + prompt variance) — not a general coding-task or SWE-bench claim. Skill/customize may not separate when the entry doc already teaches the correct rule.
|
|
67
|
+
|
|
68
|
+
### Industry context
|
|
69
|
+
|
|
70
|
+
Not a Skeleton measurement — broader context research points the same direction: more context isn’t free. A [2026 study of repo-level context files](https://doi.org/10.48550/arxiv.2602.11988) (438 tasks) found human-written files helped ~4% on average, generated files hurt ~3%, and both raised inference cost >20%. A [2025 METR trial](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/) found experienced OSS developers took 19% longer with early-2025 AI tools while believing they were faster. Those papers motivate small instructions and explicit ownership; they do **not** measure this tool.
|
|
45
71
|
|
|
46
72
|
## Quick start
|
|
47
73
|
|
|
@@ -71,7 +97,7 @@ Flag details: [install](docs/developer/install.md).
|
|
|
71
97
|
- **Prose policy** (optional plugins) — YAML pattern rules; idle with no plugins
|
|
72
98
|
- **Shell / JSON syntax** — lightweight checks on changed `.sh` and `.json` files
|
|
73
99
|
|
|
74
|
-
Skeleton doesn't replace your code gates. Keep TypeScript, Python, Nx, pytest,
|
|
100
|
+
Skeleton doesn't replace your code gates. Keep TypeScript, Python, Nx, pytest, and the rest in the repo that owns them.
|
|
75
101
|
|
|
76
102
|
## The `.skeleton/` contract
|
|
77
103
|
|
|
@@ -96,7 +122,7 @@ Register it:
|
|
|
96
122
|
skeleton register docs/developer/api.md
|
|
97
123
|
```
|
|
98
124
|
|
|
99
|
-
Synced skills stay pristine. Project overrides live in `.skeleton/customize/<slug>.md` and inject when the skill is read
|
|
125
|
+
Synced skills stay pristine. Project overrides live in `.skeleton/customize/<slug>.md` and inject when the skill is read — no editing copied `SKILL.md` files.
|
|
100
126
|
|
|
101
127
|
## Commands
|
|
102
128
|
|
|
@@ -112,14 +138,14 @@ skeleton customize resolve <slug>
|
|
|
112
138
|
|
|
113
139
|
**Validate changed** routes git diffs to the right audit:
|
|
114
140
|
|
|
115
|
-
| Path | Action
|
|
116
|
-
| ----------------------------------------------------------------------------------- |
|
|
117
|
-
| Docs in scan perimeter | path-scoped audit
|
|
118
|
-
| Owned skill bodies (`SKILL.md` trees) | exit 1 → run `audit skills`
|
|
119
|
-
| Foreign / lockfile-synced skill bodies | skip → lint in the owning skills/toolbox repo
|
|
120
|
-
| `.sh`, `.bash`, `.zsh` | shellcheck or `bash -n`
|
|
121
|
-
| Other `.json` | JSONC-tolerant syntax check
|
|
122
|
-
| `.ts`, `.tsx`, `.js`, `.jsx`, `.mjs`, `.cjs`, `.py`, `package.json`, `project.json` | skip (exits 1 if all skip)
|
|
141
|
+
| Path | Action |
|
|
142
|
+
| ----------------------------------------------------------------------------------- | --------------------------------------------- |
|
|
143
|
+
| Docs in scan perimeter | path-scoped audit |
|
|
144
|
+
| Owned skill bodies (`SKILL.md` trees) | exit 1 → run `audit skills` |
|
|
145
|
+
| Foreign / lockfile-synced skill bodies | skip → lint in the owning skills/toolbox repo |
|
|
146
|
+
| `.sh`, `.bash`, `.zsh` | shellcheck or `bash -n` |
|
|
147
|
+
| Other `.json` | JSONC-tolerant syntax check |
|
|
148
|
+
| `.ts`, `.tsx`, `.js`, `.jsx`, `.mjs`, `.cjs`, `.py`, `package.json`, `project.json` | skip (exits 1 if all skip) |
|
|
123
149
|
|
|
124
150
|
Pre-commit: `skeleton validate changed --staged` (path-scoped, fast).
|
|
125
151
|
|
|
@@ -127,8 +153,6 @@ CI: `skeleton validate changed --base origin/main` (global rules first, then cha
|
|
|
127
153
|
|
|
128
154
|
## Ecosystem
|
|
129
155
|
|
|
130
|
-
The model is simple:
|
|
131
|
-
|
|
132
156
|
| Layer | Role |
|
|
133
157
|
| ----------------- | ---------------------------------------------------------------------------------------------- |
|
|
134
158
|
| **Skeleton** | Defines and checks the SSOT contract |
|
|
@@ -160,6 +184,8 @@ bun run check
|
|
|
160
184
|
|
|
161
185
|
`bun run check` = lint + test + typecheck + build + `audit:self`.
|
|
162
186
|
|
|
163
|
-
`validate:changed` is docs/config only for path-scoped work — it skips code/config extensions (see table above). Owned skill-body edits need `audit skills`. All-skip, owned skill paths (alone or mixed with docs), and missing paths exit non-zero.
|
|
187
|
+
`validate:changed` is docs/config only for path-scoped work — it skips code/config extensions (see table above). Owned skill-body edits need `audit skills`. All-skip, owned skill paths (alone or mixed with docs), and missing paths exit non-zero.
|
|
188
|
+
|
|
189
|
+
For code: `bun test`, `bun run typecheck`, `bun run build`.
|
|
164
190
|
|
|
165
191
|
Optional: `brew install pre-commit` (or `pipx install pre-commit`), then `pre-commit install` to wire `.pre-commit-config.yaml`.
|
package/dist/cli.js
CHANGED
|
@@ -17385,6 +17385,11 @@ function collectCoverageCandidateFiles(root, exclude) {
|
|
|
17385
17385
|
}
|
|
17386
17386
|
return [...files];
|
|
17387
17387
|
}
|
|
17388
|
+
function excludeForeignSkillDocMetaPaths(docMetaPaths, skillIndex) {
|
|
17389
|
+
if (!skillIndex)
|
|
17390
|
+
return docMetaPaths;
|
|
17391
|
+
return docMetaPaths.filter((rel) => !isForeignSkillPath(rel, skillIndex));
|
|
17392
|
+
}
|
|
17388
17393
|
function collectDocMetaPaths(config, root, registryPaths, skillIndex) {
|
|
17389
17394
|
const paths = [];
|
|
17390
17395
|
for (const abs of expandPatterns(root, ["docs/*/README.md"], mergedExcludes(config))) {
|
|
@@ -17409,7 +17414,7 @@ function collectDocMetaPaths(config, root, registryPaths, skillIndex) {
|
|
|
17409
17414
|
paths.push(normalizeRelPath(relative4(root, abs)));
|
|
17410
17415
|
}
|
|
17411
17416
|
}
|
|
17412
|
-
return [...new Set(paths)];
|
|
17417
|
+
return excludeForeignSkillDocMetaPaths([...new Set(paths)], skillIndex);
|
|
17413
17418
|
}
|
|
17414
17419
|
function validateScanRoots(config, root) {
|
|
17415
17420
|
const missing = [];
|
|
@@ -17419,11 +17424,11 @@ function validateScanRoots(config, root) {
|
|
|
17419
17424
|
}
|
|
17420
17425
|
return missing;
|
|
17421
17426
|
}
|
|
17422
|
-
function filterDocMetaPaths(docMetaPaths, paths) {
|
|
17427
|
+
function filterDocMetaPaths(docMetaPaths, paths, skillIndex) {
|
|
17423
17428
|
if (paths.length === 0)
|
|
17424
|
-
return docMetaPaths;
|
|
17429
|
+
return excludeForeignSkillDocMetaPaths(docMetaPaths, skillIndex);
|
|
17425
17430
|
const normalizedPaths = paths.map((path) => normalizeRelPath(path));
|
|
17426
|
-
return docMetaPaths.filter((rel) => normalizedPaths.some((path) => rel === path || rel.startsWith(`${path}/`)));
|
|
17431
|
+
return excludeForeignSkillDocMetaPaths(docMetaPaths.filter((rel) => normalizedPaths.some((path) => rel === path || rel.startsWith(`${path}/`))), skillIndex);
|
|
17427
17432
|
}
|
|
17428
17433
|
function filterToPaths(files, paths, root) {
|
|
17429
17434
|
const normalizedPaths = paths.map((path) => normalizeRelPath(path));
|
|
@@ -17516,7 +17521,7 @@ function createContext(options = {}) {
|
|
|
17516
17521
|
root,
|
|
17517
17522
|
config,
|
|
17518
17523
|
files,
|
|
17519
|
-
docMetaPaths: options.paths && options.paths.length > 0 ? filterDocMetaPaths(allDocMetaPaths, options.paths) : allDocMetaPaths,
|
|
17524
|
+
docMetaPaths: options.paths && options.paths.length > 0 ? filterDocMetaPaths(allDocMetaPaths, options.paths, skillIndex) : allDocMetaPaths,
|
|
17520
17525
|
registryPaths: registry.paths,
|
|
17521
17526
|
registryHasTableHeader: registry.hasTableHeader,
|
|
17522
17527
|
retiredSkills: new Set(retiredSkills(config)),
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@csark0812/skeleton",
|
|
3
|
-
"version": "1.5.
|
|
3
|
+
"version": "1.5.5",
|
|
4
4
|
"description": "SSOT audit CLI for agent harness repos",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -47,6 +47,15 @@
|
|
|
47
47
|
"check": "bun run lint && bun test && bun run typecheck && bun run build && bun run audit:self",
|
|
48
48
|
"start": "bun src/cli.ts --help",
|
|
49
49
|
"dev": "bun src/cli.ts --help",
|
|
50
|
+
"agent:test:live": "agent-test --suites-dir agent-suites --live --fail-on=behavior",
|
|
51
|
+
"agent:test:live:compare": "agent-test --suites-dir agent-suites --live --compare-pairs skeleton-clean:skeleton-messy --fail-on=behavior --out-dir \"${TMPDIR:-/tmp}/skeleton-compare\"",
|
|
52
|
+
"agent:test:live:debug": "agent-test --suites-dir agent-suites --live --fail-on=behavior --debug",
|
|
53
|
+
"agent:test:doctor": "agent-test --doctor",
|
|
54
|
+
"agent:test:validate": "agent-test --validate-only --validate-paths --suites-dir agent-suites",
|
|
55
|
+
"agent:evidence:aggregate": "bun scripts/agent-evidence/aggregate-compares.ts",
|
|
56
|
+
"agent:evidence:charts": "bun scripts/agent-evidence/render-charts.ts",
|
|
57
|
+
"agent:evidence:excerpt": "bun scripts/agent-evidence/excerpt-transcript.ts",
|
|
58
|
+
"agent:evidence:archive": "bash scripts/agent-evidence/archive-run.sh",
|
|
50
59
|
"prepack": "bun test && bun run typecheck && bun run build"
|
|
51
60
|
},
|
|
52
61
|
"dependencies": {
|
|
@@ -60,6 +69,8 @@
|
|
|
60
69
|
},
|
|
61
70
|
"devDependencies": {
|
|
62
71
|
"@biomejs/biome": "^2.5.3",
|
|
72
|
+
"@cursor/sdk": "^1.0.23",
|
|
73
|
+
"@post-print/agent-test": "^0.2.7",
|
|
63
74
|
"@types/bun": "^1.3.8",
|
|
64
75
|
"@types/github-slugger": "^2.0.0",
|
|
65
76
|
"typescript": "~7.0.2"
|