mdkg 0.1.2 → 0.1.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. package/CHANGELOG.md +51 -0
  2. package/README.md +31 -15
  3. package/dist/cli.js +182 -85
  4. package/dist/commands/archive.js +20 -8
  5. package/dist/commands/bundle.js +7 -7
  6. package/dist/commands/capability.js +118 -4
  7. package/dist/commands/checkpoint.js +31 -5
  8. package/dist/commands/doctor.js +61 -24
  9. package/dist/commands/index.js +12 -23
  10. package/dist/commands/init.js +7 -1
  11. package/dist/commands/list.js +1 -1
  12. package/dist/commands/new.js +33 -7
  13. package/dist/commands/next.js +1 -1
  14. package/dist/commands/node_card.js +1 -1
  15. package/dist/commands/pack.js +1 -1
  16. package/dist/commands/search.js +1 -1
  17. package/dist/commands/show.js +1 -1
  18. package/dist/commands/subgraph.js +312 -0
  19. package/dist/commands/task.js +21 -7
  20. package/dist/commands/upgrade.js +51 -4
  21. package/dist/commands/validate.js +12 -6
  22. package/dist/commands/work.js +44 -12
  23. package/dist/core/config.js +110 -39
  24. package/dist/graph/capabilities_index_cache.js +2 -2
  25. package/dist/graph/index_cache.js +14 -14
  26. package/dist/graph/indexer.js +1 -1
  27. package/dist/graph/reindex.js +46 -0
  28. package/dist/graph/skills_index_cache.js +2 -2
  29. package/dist/graph/sqlite_index.js +293 -0
  30. package/dist/graph/{bundle_imports.js → subgraphs.js} +216 -142
  31. package/dist/graph/visibility.js +3 -3
  32. package/dist/init/AGENT_START.md +5 -1
  33. package/dist/init/CLI_COMMAND_MATRIX.md +21 -7
  34. package/dist/init/README.md +20 -10
  35. package/dist/init/config.json +6 -2
  36. package/dist/init/core/rule-1-mdkg-conventions.md +2 -1
  37. package/dist/init/core/rule-3-cli-contract.md +32 -24
  38. package/dist/init/core/rule-4-repo-safety-and-ignores.md +28 -12
  39. package/dist/init/core/rule-5-release-and-versioning.md +4 -3
  40. package/dist/init/init-manifest.json +10 -10
  41. package/dist/init/skills/default/verify-close-and-checkpoint/SKILL.md +1 -1
  42. package/dist/util/argparse.js +2 -0
  43. package/dist/util/atomic.js +44 -0
  44. package/dist/util/lock.js +72 -0
  45. package/package.json +13 -9
  46. package/dist/commands/bundle_import.js +0 -243
@@ -2,6 +2,8 @@
2
2
 
3
3
  This repository is initialized for mdkg.
4
4
 
5
+ mdkg is pre-v1 public alpha software. Graph, cache, bundle, and DAL contracts may change quickly before v1.
6
+
5
7
  ## Layout
6
8
 
7
9
  - `core/`: rules, operating guide, and pinned docs
@@ -10,7 +12,7 @@ This repository is initialized for mdkg.
10
12
  - `templates/`: default node templates
11
13
  - `archive/`: sidecar metadata and deterministic compressed source/artifact caches
12
14
  - `bundles/`: optional committed full graph snapshot bundles
13
- - `index/`: generated index cache (do not commit)
15
+ - `index/`: generated JSON caches plus optional commit-eligible `mdkg.sqlite`
14
16
  - `pack/`: generated context packs (do not commit)
15
17
 
16
18
  ## Next Commands
@@ -24,11 +26,11 @@ mdkg pack <id>
24
26
  mdkg capability search "..."
25
27
  mdkg archive list
26
28
  mdkg bundle create --profile private
27
- mdkg bundle import list --json
29
+ mdkg subgraph list --json
28
30
  mdkg validate
29
31
  ```
30
32
 
31
- This repo is already initialized. Use `mdkg upgrade` to preview safe scaffold updates, `mdkg new` to create work, `mdkg search`/`mdkg show` to inspect graph state, `mdkg capability ...` to inspect cached skill/spec/work/core/design capabilities, `mdkg archive ...` to register source/artifact sidecars, `mdkg work ...` to create work contract/order/receipt semantic mirrors, `mdkg bundle ...` to create full graph snapshot bundles and read-only child graph imports, `mdkg pack <id>` to build deterministic context, and `mdkg validate` before closeout.
33
+ This repo is already initialized. Use `mdkg upgrade` to preview safe scaffold updates, `mdkg new` to create work, `mdkg search`/`mdkg show` to inspect graph state, `mdkg capability ...` to inspect cached skill/spec/work/core/design capabilities, `mdkg capability resolve ...` to rank local and subgraph capabilities, `mdkg archive ...` to register source/artifact sidecars, `mdkg work ...` to create work contract/order/receipt semantic mirrors, `mdkg bundle ...` to create full graph snapshot bundles, `mdkg subgraph ...` to register read-only child graph planning views, `mdkg pack <id>` to build deterministic context, and `mdkg validate` before closeout.
32
34
 
33
35
  Agent workflow docs can use semantic ids:
34
36
 
@@ -51,10 +53,17 @@ Read `AGENT_START.md` first when this repo includes it.
51
53
 
52
54
  Ensure ignore files include:
53
55
 
54
- - `.mdkg/index/`
56
+ - `.mdkg/index/*.json`
57
+ - `.mdkg/index/*.tmp`
58
+ - `.mdkg/index/write.lock/`
59
+ - `.mdkg/index/*.sqlite-wal`
60
+ - `.mdkg/index/*.sqlite-shm`
61
+ - `.mdkg/index/*.sqlite-journal`
55
62
  - `.mdkg/pack/`
56
63
  - `.mdkg/archive/**/source/`
57
64
 
65
+ Fresh mdkg workspaces default to `index.backend: sqlite`; `.mdkg/index/mdkg.sqlite` is a rebuildable cache and may be committed when the repo intentionally tracks it and it stays reasonably small.
66
+
58
67
  Recommended:
59
68
 
60
69
  ```bash
@@ -78,16 +87,17 @@ mdkg bundle create --profile private
78
87
  mdkg bundle verify .mdkg/bundles/private/all.mdkg.zip
79
88
  ```
80
89
 
81
- Use this as a pre-commit recommendation only when the repo tracks archive caches or `.mdkg/bundles/`. Private bundles are local graph transport artifacts and may be tracked in private repos when configured. Public bundles require selected workspaces with `visibility: public` and fail closed when public records reference private graph, archive, or imported records.
90
+ Use this as a pre-commit recommendation only when the repo tracks archive caches or `.mdkg/bundles/`. Private bundles are local graph transport artifacts and may be tracked in private repos when configured. Public bundles require selected workspaces with `visibility: public` and fail closed when public records reference private graph, archive, or subgraph records.
82
91
 
83
- Register child bundle snapshots as read-only imports with:
92
+ Register child bundle snapshots as read-only subgraphs with:
84
93
 
85
94
  ```bash
86
- mdkg bundle import add child_repo child-repo/.mdkg/bundles/private/all.mdkg.zip --source-path child-repo
87
- mdkg bundle import verify child_repo --json
95
+ mdkg subgraph add child_repo child-repo/.mdkg/bundles/private/all.mdkg.zip --source-path child-repo
96
+ mdkg capability resolve "child capability" --json
97
+ mdkg subgraph verify child_repo --json
88
98
  ```
89
99
 
90
- Imported nodes use the import alias as their qid prefix and can be inspected or packed, but mutations must happen in the owning child repo.
100
+ Subgraph nodes use the subgraph alias as their qid prefix and can be inspected or packed, but mutations must happen in the owning child repo.
91
101
 
92
102
  ## Archive and Work Mirrors
93
103
 
@@ -109,7 +119,7 @@ mdkg work receipt new "example receipt" --id receipt.example-1 --work-order-id o
109
119
  ```
110
120
 
111
121
  Receipt statuses are `recorded`, `verified`, `rejected`, and `superseded`.
112
- Update and artifact commands accept local ids or local qids; imported bundle qids are read-only and must be changed in their source workspace.
122
+ Update and artifact commands accept local ids or local qids; subgraph qids are read-only and must be changed in their source workspace.
113
123
 
114
124
  Production orders, receipts, feedback, disputes, payments, ledgers, marketplace inventory, fulfillment records, and execution state remain canonical outside mdkg. mdkg stores committed semantic mirrors and reviewable evidence. Do not store raw secrets, credentials, live payment state, ledger mutations, canonical marketplace state, or bulky raw payloads in these mirrors.
115
125
 
@@ -8,7 +8,11 @@
8
8
  "index": {
9
9
  "auto_reindex": true,
10
10
  "tolerant": false,
11
- "global_index_path": ".mdkg/index/global.json"
11
+ "backend": "sqlite",
12
+ "global_index_path": ".mdkg/index/global.json",
13
+ "sqlite_path": ".mdkg/index/mdkg.sqlite",
14
+ "sqlite_commit_warning_bytes": 52428800,
15
+ "lock_timeout_ms": 10000
12
16
  },
13
17
  "capabilities": {
14
18
  "cache_path": ".mdkg/index/capabilities.json"
@@ -17,7 +21,7 @@
17
21
  "output_dir": ".mdkg/bundles",
18
22
  "default_profile": "private"
19
23
  },
20
- "bundle_imports": {},
24
+ "subgraphs": {},
21
25
  "pack": {
22
26
  "default_depth": 2,
23
27
  "default_edges": [
@@ -229,8 +229,9 @@ The cache is enabled by default.
229
229
 
230
230
  - Root global index lives at `.mdkg/index/global.json`
231
231
  - Root skills index lives at `.mdkg/index/skills.json`
232
+ - Root SQLite access cache lives at `.mdkg/index/mdkg.sqlite` when `index.backend` is `sqlite`.
232
233
  - Index is rebuilt automatically when stale unless disabled by flag/config.
233
- - `.mdkg/index/` is generated and MUST be gitignored.
234
+ - Generated JSON index/temp/lock files are ignored. `.mdkg/index/mdkg.sqlite` is rebuildable and may be committed only by explicit repo policy.
234
235
 
235
236
  ## Safety guidance (high level)
236
237
 
@@ -69,9 +69,9 @@ Workspaces are registered in `.mdkg/config.json`.
69
69
 
70
70
  Qualified IDs may be used as input:
71
71
  - `<ws>:<id>` (example: `e2e:task-12`)
72
- - Imported bundle nodes use the same qualified form with the import alias:
73
- - `<import-alias>:<id>` (example: `agent_image:work.generate-image`)
74
- - imported nodes are read-only planning context and MUST NOT be selected by local mutation commands
72
+ - Subgraph nodes use the same qualified form with the subgraph alias:
73
+ - `<subgraph-alias>:<id>` (example: `agent_image:work.generate-image`)
74
+ - subgraph nodes are read-only planning context and MUST NOT be selected by local mutation commands
75
75
 
76
76
  If a user provides an unqualified ID and it is ambiguous globally:
77
77
  - mdkg MUST error and suggest qualified IDs.
@@ -124,7 +124,7 @@ If a user provides an unqualified ID and it is ambiguous globally:
124
124
  - `.agents/skills/`
125
125
  - `.claude/skills/`
126
126
  - deterministic `core.md` pin insertion (`rule-soul`, then `rule-human`)
127
- - ignore policy for `.mdkg/index/`, `.mdkg/pack/`, and raw archive source copies under `.mdkg/archive/**/source/`
127
+ - ignore policy for generated JSON index/temp/lock files, `.mdkg/pack/`, and raw archive source copies under `.mdkg/archive/**/source/`
128
128
  - mirrored skills are append-focused outputs:
129
129
  - `.mdkg/skills/` remains canonical
130
130
  - unrelated existing folders under `.agents/skills/` and `.claude/skills/` are preserved
@@ -145,7 +145,8 @@ If a user provides an unqualified ID and it is ambiguous globally:
145
145
  - rebuild global cache `.mdkg/index/global.json`
146
146
  - rebuild skills cache `.mdkg/index/skills.json` from `.mdkg/skills/<slug>/SKILL.md`
147
147
  - rebuild capability cache `.mdkg/index/capabilities.json` from skills, `SPEC.md`, `WORK.md`, core docs, and design docs
148
- - rebuild bundle import projection cache `.mdkg/index/imports.json` when bundle imports are configured
148
+ - rebuild subgraph projection cache `.mdkg/index/subgraphs.json` when subgraphs are configured
149
+ - rebuild SQLite access cache `.mdkg/index/mdkg.sqlite` when `index.backend` is `sqlite`
149
150
  - tolerate `.mdkg/skills/<slug>/SKILLS.md` on read with warning
150
151
  - fail validation if both `SKILL.md` and `SKILLS.md` exist in one skill directory
151
152
  - strict by default (fails on invalid frontmatter)
@@ -182,10 +183,10 @@ Common flags:
182
183
  - `mdkg search "<query>" [--type <type>] [--status <status>] [--ws <alias>] [--tags <tag,tag,...>] [--tags-mode any|all] [--json|--xml|--toon|--md]`
183
184
  - search SHOULD match on IDs, titles, tags, path tokens, and searchable frontmatter lists (`links`, `artifacts`, `refs`, `aliases`)
184
185
  - `mdkg list [--type <type>] [--status <status>] [--ws <alias>] [--epic <id>] [--blocked] [--priority <n>] [--tags <tag,tag,...>] [--tags-mode any|all] [--json|--xml|--toon|--md]`
185
- - enabled bundle imports are included in `show`, `search`, `list`, `pack`, and `capability` reads by default:
186
- - imported nodes surface `source.imported: true` in JSON output
187
- - human output labels imported nodes as read-only and stale when applicable
188
- - stale imports warn during planning reads but remain usable
186
+ - enabled subgraphs are included in `show`, `search`, `list`, `pack`, and `capability` reads by default:
187
+ - subgraph nodes surface `source.imported: true` and `source.subgraph_alias` in JSON output
188
+ - human output labels subgraph nodes as read-only and stale when applicable
189
+ - stale subgraphs warn during planning reads but remain usable
189
190
  - skills are first-class under `mdkg skill ...` only:
190
191
  - `mdkg skill list [--tags <tag,tag,...>] [--tags-mode any|all] [--json|--xml|--toon|--md]`
191
192
  - `mdkg skill show <slug> [--meta] [--json|--xml|--toon|--md]`
@@ -196,7 +197,9 @@ Common flags:
196
197
  - `mdkg capability list [--kind <skill|spec|work|core|design>] [--visibility <private|internal|public>] [--json]`
197
198
  - `mdkg capability search "<query>" [--kind <kind>] [--visibility <level>] [--json]`
198
199
  - `mdkg capability show <id-or-qid-or-slug> [--json]`
200
+ - `mdkg capability resolve [query] [--requires <capability>] [--fresh-only] [--json]`
199
201
  - capability records are read-only derived cache entries, not source of truth
202
+ - `resolve` ranks local and subgraph capability candidates deterministically and degrades stale subgraphs unless `--fresh-only` is supplied
200
203
  - normal task, epic, feat, bug, test, and checkpoint nodes are not capability records
201
204
  - archives are first-class sidecar nodes under `mdkg archive ...`:
202
205
  - `mdkg archive add <file> [--id <archive.id>] [--kind source|artifact] [--visibility private|internal|public] [--title <title>] [--refs <...>] [--relates <...>] [--json]`
@@ -214,21 +217,26 @@ Common flags:
214
217
  - `mdkg bundle verify [bundle-path] [--json]`
215
218
  - `mdkg bundle show <bundle-path> [--json]`
216
219
  - `mdkg bundle list [--json]`
217
- - `mdkg bundle import add <alias> <bundle-path> [--visibility private|internal|public] [--profile private|public] [--source-path <path>] [--source-repo <ref>] [--max-stale-seconds <seconds>] [--json]`
218
- - `mdkg bundle import list [--json]`
219
- - `mdkg bundle import rm <alias> [--json]`
220
- - `mdkg bundle import enable <alias> [--json]`
221
- - `mdkg bundle import disable <alias> [--json]`
222
- - `mdkg bundle import verify [alias|--all] [--json]`
223
220
  - bundles are explicit transport artifacts and are not rewritten by `mdkg index`
224
221
  - default output is `.mdkg/bundles/<profile>/<workspace-or-all>.mdkg.zip`
225
222
  - public bundles must fail closed when public records reference private graph or archive records
226
- - public bundles must fail closed when public records reference private/internal imported graph records
227
- - bundle imports are read-only projected graph views; child repos remain owners of real mutations and commits
228
- - `bundle import verify` exits nonzero for stale, missing, corrupt, profile-mismatched, or duplicate-id imports
229
- - public bundle creation must not re-export imported child graph content and must fail if public local nodes reference private/internal imports
230
- - public/internal imports require `expected_profile: public`; private bundle profiles cannot be promoted through import visibility
223
+ - public bundles must fail closed when public records reference private/internal subgraph records
224
+ - public bundle creation must not re-export subgraph content and must fail if public local nodes reference private/internal subgraphs
231
225
  - `mdkg pack --visibility public|internal|private` records explicit pack visibility and filters public/internal packs through the same fail-closed policy
226
+ - subgraph orchestration lives under `mdkg subgraph ...`:
227
+ - `mdkg subgraph add <alias> <bundle-path> [--visibility private|internal|public] [--profile private|public] [--source-path <path>] [--source-repo <ref>] [--max-stale-seconds <seconds>] [--json]`
228
+ - `mdkg subgraph list [--json]`
229
+ - `mdkg subgraph show <alias> [--json]`
230
+ - `mdkg subgraph rm <alias> [--json]`
231
+ - `mdkg subgraph enable <alias> [--json]`
232
+ - `mdkg subgraph disable <alias> [--json]`
233
+ - `mdkg subgraph verify [alias|--all] [--json]`
234
+ - `mdkg subgraph refresh [alias|--all] [--json]`
235
+ - subgraphs are read-only projected graph views; child repos remain owners of real mutations and commits
236
+ - `subgraph refresh` reloads configured bundle sources only and never builds or mutates child repos
237
+ - `subgraph verify` exits nonzero for stale, missing, corrupt, profile-mismatched, or duplicate-id subgraphs
238
+ - public/internal subgraphs require `expected_profile: public`; private bundle profiles cannot be promoted through subgraph visibility
239
+ - legacy `mdkg bundle import ...` exits with guidance to run `mdkg upgrade --apply` and use `mdkg subgraph ...`
232
240
  - work lifecycle helpers live under `mdkg work ...`:
233
241
  - `mdkg work contract new "<title>" --id <work.id> --agent-id <agent.id> --kind <kind> --inputs <...> --outputs <...> [--required-capabilities <...>] [--pricing-model <...>] [--json]`
234
242
  - `mdkg work order new "<title>" --id <order.id> --work-id <work.id> --requester <ref> [--request-ref <ref>] [--input-refs <...>] [--requested-outputs <...>] [--json]`
@@ -239,7 +247,7 @@ Common flags:
239
247
  - these commands mutate mdkg semantic mirror files only; production order, receipt, feedback, dispute, payment, ledger, marketplace inventory, fulfillment, and execution state remains canonical outside mdkg
240
248
  - work mirrors must not store raw secrets, credentials, live payment state, ledger mutations, or canonical marketplace state
241
249
  - `artifact://...` refs identify external/runtime-managed artifacts; `archive://...` refs identify committed mdkg archive sidecars
242
- - update and artifact commands accept local ids or local qids; imported bundle qids are read-only and must be changed in their source workspace
250
+ - update and artifact commands accept local ids or local qids; subgraph qids are read-only and must be changed in their source workspace
243
251
  - discovery/show output flags are mutually exclusive; text mode remains the default when none are supplied
244
252
 
245
253
  ### Task lifecycle mutation
@@ -250,7 +258,7 @@ Common flags:
250
258
  - supports additive list mutation for `artifacts`, `links`, `refs`, `skills`, `tags`, and `blocked_by`
251
259
  - supports scalar replacement for `status` and `priority`
252
260
  - `--clear-blocked-by` resets blockers before optional re-add
253
- - imported bundle qids fail with an explicit read-only import error
261
+ - subgraph qids fail with an explicit read-only subgraph error
254
262
  - `mdkg task done <id-or-qid> [--checkpoint "<title>"] [...]`
255
263
  - supports `task`, `bug`, and `test` nodes only
256
264
  - sets `status: done`
@@ -307,8 +315,8 @@ Common flags:
307
315
  - `mdkg validate`
308
316
  - strict frontmatter + graph integrity checks (exit code 2 on failure)
309
317
  - validates optional node->skill references
310
- - validates configured bundle imports and fails on missing/corrupt enabled bundles, malformed import config, duplicate projected ids, and invalid import refs
311
- - warns, but does not fail, on stale imports
318
+ - validates configured subgraphs and fails on missing/corrupt enabled bundles, malformed subgraph config, duplicate projected ids, and invalid subgraph refs
319
+ - warns, but does not fail, on stale subgraphs
312
320
  - validates optional `.mdkg/work/events/events.jsonl` record shape when file exists
313
321
  - warns when `.agents/skills/` or `.claude/skills/` drift from canonical `.mdkg/skills/`
314
322
  - `mdkg format`
@@ -21,7 +21,8 @@ mdkg content may contain sensitive notes and internal project planning. This rul
21
21
 
22
22
  - `.mdkg/` must not be shipped to production deployments.
23
23
  - `.mdkg/` must not be published to npm.
24
- - `.mdkg/index/` must never be committed.
24
+ - Generated JSON index, temp, lock, WAL, SHM, and journal files under `.mdkg/index/` must not be committed.
25
+ - `.mdkg/index/mdkg.sqlite` is a rebuildable access cache and may be committed when the repo intentionally tracks it and it stays reasonably small.
25
26
  - `.mdkg/bundles/` may be committed only when the repo intentionally tracks private or public snapshot bundles.
26
27
 
27
28
  ## Git ignore requirements
@@ -30,13 +31,22 @@ The repo MUST ignore at minimum:
30
31
 
31
32
  - `node_modules/`
32
33
  - `dist/`
33
- - `.mdkg/index/`
34
+ - `.mdkg/index/*.json`
35
+ - `.mdkg/index/*.tmp`
36
+ - `.mdkg/index/write.lock/`
37
+ - `.mdkg/index/*.sqlite-wal`
38
+ - `.mdkg/index/*.sqlite-shm`
39
+ - `.mdkg/index/*.sqlite-journal`
34
40
  - `.mdkg/pack/`
35
41
  - `.mdkg/archive/**/source/`
36
42
 
37
43
  Recommended `.gitignore` entries:
38
- - `.mdkg/index/`
39
- - `.mdkg/index/**`
44
+ - `.mdkg/index/*.json`
45
+ - `.mdkg/index/*.tmp`
46
+ - `.mdkg/index/write.lock/`
47
+ - `.mdkg/index/*.sqlite-wal`
48
+ - `.mdkg/index/*.sqlite-shm`
49
+ - `.mdkg/index/*.sqlite-journal`
40
50
  - `.mdkg/pack/`
41
51
  - `.mdkg/archive/**/source/`
42
52
 
@@ -60,7 +70,12 @@ Additional belt-and-suspenders:
60
70
  If the repo is containerized:
61
71
  - `.dockerignore` SHOULD exclude:
62
72
  - `.mdkg/`
63
- - `.mdkg/index/`
73
+ - `.mdkg/index/*.json`
74
+ - `.mdkg/index/*.tmp`
75
+ - `.mdkg/index/write.lock/`
76
+ - `.mdkg/index/*.sqlite-wal`
77
+ - `.mdkg/index/*.sqlite-shm`
78
+ - `.mdkg/index/*.sqlite-journal`
64
79
  - `node_modules/`
65
80
  - `dist/` (if built in container)
66
81
  - any other local artifacts
@@ -72,8 +87,8 @@ For application builds:
72
87
 
73
88
  `mdkg init` updates ignore files by default for safety:
74
89
 
75
- - `.gitignore` appends `.mdkg/index/`, `.mdkg/pack/`
76
- - `.npmignore` appends `.mdkg/`, `.mdkg/index/`, `.mdkg/pack/`
90
+ - `.gitignore` appends generated index cache/temp/lock patterns, `.mdkg/pack/`, and raw archive source ignores.
91
+ - `.npmignore` appends `.mdkg/`, generated index cache/temp/lock patterns, and `.mdkg/pack/`.
77
92
  - `--no-update-ignores` disables these default writes
78
93
 
79
94
  Explicit flags remain available and take precedence:
@@ -84,16 +99,16 @@ Explicit flags remain available and take precedence:
84
99
 
85
100
  ## Index safety
86
101
 
87
- - `.mdkg/index/` is generated.
88
- - Index files may contain extracted metadata and could expose sensitive strings.
89
- - Index files MUST be ignored from git.
102
+ - `.mdkg/index/` contains generated caches.
103
+ - JSON index files may contain extracted metadata and could expose sensitive strings; they MUST be ignored from git.
104
+ - `.mdkg/index/mdkg.sqlite` contains the same rebuildable access data and may be committed only by explicit repo policy; `mdkg doctor` warns when it exceeds `index.sqlite_commit_warning_bytes`.
90
105
  - Index rebuild should be deterministic and safe to regenerate at any time.
91
106
 
92
107
  ## Bundle safety
93
108
 
94
109
  - `.mdkg/bundles/` stores explicit snapshot artifacts and is not ignored by default.
95
110
  - Private bundles may include sensitive authored mdkg content and should stay in private repos.
96
- - Public bundles must be created with `mdkg bundle create --profile public` so private graph, archive, and imported bundle refs fail closed.
111
+ - Public bundles must be created with `mdkg bundle create --profile public` so private graph, archive, and subgraph refs fail closed.
97
112
  - Public-safe packs must be created with `mdkg pack <id> --visibility public`; internal-safe packs use `--visibility internal`. These filters do not redact Markdown body text.
98
113
  - Bundle ZIPs must exclude `.mdkg/pack/`, existing `.mdkg/index/`, nested `.mdkg/bundles/`, and raw `.mdkg/archive/**/source/` files.
99
114
  - Repos that track archive caches or bundles should refresh in this order before commit: `mdkg archive compress --all`, `mdkg archive verify --json`, `mdkg bundle create --profile private`, then bundle verify.
@@ -115,7 +130,8 @@ Workspace-local `.mdkg/` directories (near code) should follow the same rules:
115
130
 
116
131
  ## Summary checklist
117
132
 
118
- - ✅ `.mdkg/index/` ignored
133
+ - ✅ generated JSON index/temp/lock files ignored
134
+ - ✅ `.mdkg/index/mdkg.sqlite` committed only by explicit repo policy
119
135
  - ✅ event logs are committed by default unless a repo chooses to ignore them manually
120
136
  - ✅ npm publishes only `dist/`, `README.md`, `LICENSE`
121
137
  - ✅ optional `.npmignore` excludes `.mdkg/`
@@ -37,14 +37,15 @@ The npm package MUST include:
37
37
 
38
38
  It MUST NOT include:
39
39
  - `.mdkg/` docs
40
- - `.mdkg/index/`
40
+ - generated `.mdkg/index/` caches
41
41
  - source code (optional; can be included later, but not required)
42
42
 
43
43
  ## Release checklist (v1)
44
44
 
45
45
  1) Ensure clean working tree
46
46
  - no uncommitted changes
47
- - all `.mdkg/index/` ignored
47
+ - generated JSON index/temp/lock files ignored
48
+ - `.mdkg/index/mdkg.sqlite` either intentionally tracked or absent
48
49
 
49
50
  2) Rebuild and validate
50
51
  - run `mdkg index`
@@ -79,4 +80,4 @@ Release notes should call out:
79
80
  - new commands / flags
80
81
  - changes to pack behavior
81
82
  - changes to config schema (and how migration behaves)
82
- - new node types or template changes
83
+ - new node types or template changes
@@ -1,12 +1,12 @@
1
1
  {
2
2
  "schema_version": 1,
3
3
  "tool": "mdkg",
4
- "mdkg_version": "0.1.2",
4
+ "mdkg_version": "0.1.4",
5
5
  "files": [
6
6
  {
7
7
  "path": ".mdkg/config.json",
8
8
  "category": "config",
9
- "sha256": "8f0fd9ba020511e22d912f59c39bdd1c09706595564361694860ded3e9488e7e"
9
+ "sha256": "5d2cf5e773353a59178fe86f8528413a00a73e2879fb1e020d01b49c0715a9ce"
10
10
  },
11
11
  {
12
12
  "path": ".mdkg/core/core.md",
@@ -26,7 +26,7 @@
26
26
  {
27
27
  "path": ".mdkg/core/rule-1-mdkg-conventions.md",
28
28
  "category": "core",
29
- "sha256": "20073cd21c148f1b93a382550f835ccb2e896cc676903328557004319235c112"
29
+ "sha256": "15f886e6661046de45ad798c292d74cc1ff3cc14c8cfcc82ed0b8c35635d0477"
30
30
  },
31
31
  {
32
32
  "path": ".mdkg/core/rule-2-context-pack-rules.md",
@@ -36,17 +36,17 @@
36
36
  {
37
37
  "path": ".mdkg/core/rule-3-cli-contract.md",
38
38
  "category": "core",
39
- "sha256": "ad151ac183dea1158884c4341154118d9623392871062b8faa6cecd4bde580ba"
39
+ "sha256": "4ec25c6936eefe0857a2946b47f0b65c9481bd65f1159fd5456bec6cce9de7ea"
40
40
  },
41
41
  {
42
42
  "path": ".mdkg/core/rule-4-repo-safety-and-ignores.md",
43
43
  "category": "core",
44
- "sha256": "b4638d8b5aeae74768fb971a8844aa473177df8814b929fedff6faf865a034cc"
44
+ "sha256": "2374e8684dfb32d24d560c83c99c33ff655cfdfe6811372dc3959ec93318a6db"
45
45
  },
46
46
  {
47
47
  "path": ".mdkg/core/rule-5-release-and-versioning.md",
48
48
  "category": "core",
49
- "sha256": "4fc6648052f80b90b2efc173e3bcab16cb0bc6680bb8b04f76c66fff06ad19cd"
49
+ "sha256": "e97b00aa3ade29011f1f2b482042ec292f74aad8b1d72e2f8e691cdeca5d4f70"
50
50
  },
51
51
  {
52
52
  "path": ".mdkg/core/rule-6-templates-and-schemas.md",
@@ -61,7 +61,7 @@
61
61
  {
62
62
  "path": ".mdkg/README.md",
63
63
  "category": "mdkg_doc",
64
- "sha256": "fc85d07197da2fce189d295b6e6047de2a49b711adc74c991029faec6be385de"
64
+ "sha256": "5ee6c141c0052e78671762e69c111f7f0a5d7f79b35c8f78c5b3b41901bd6959"
65
65
  },
66
66
  {
67
67
  "path": ".mdkg/skills/build-pack-and-execute-task/SKILL.md",
@@ -76,7 +76,7 @@
76
76
  {
77
77
  "path": ".mdkg/skills/verify-close-and-checkpoint/SKILL.md",
78
78
  "category": "default_skill",
79
- "sha256": "cf9d7f01eb78a3cf669b6303c2f22c7c08529fe0181e008a7b45c5e5b70ceb49"
79
+ "sha256": "67fc3d7e3c59b53add62306624391c1a416d0c66077729d79e1be0a303538f42"
80
80
  },
81
81
  {
82
82
  "path": ".mdkg/templates/default/archive.md",
@@ -176,7 +176,7 @@
176
176
  {
177
177
  "path": "AGENT_START.md",
178
178
  "category": "startup_doc",
179
- "sha256": "6ff5915ea72d7453b2f7666e138808b6b0854123c7030bcced313a15f86c2b3a"
179
+ "sha256": "abde8671d34fcc7abd8570cf8c10b940cbe8f339edb369bd046045aa5626c0fc"
180
180
  },
181
181
  {
182
182
  "path": "AGENTS.md",
@@ -191,7 +191,7 @@
191
191
  {
192
192
  "path": "CLI_COMMAND_MATRIX.md",
193
193
  "category": "startup_doc",
194
- "sha256": "f852a4f0a99c87aeb123a839c2902798ffdf5bf8012b4f367cb9c85905473ae7"
194
+ "sha256": "14deecded057a99284d2dc147855d12dedffec049da51fed3b1ebdae3105d537"
195
195
  },
196
196
  {
197
197
  "path": "llms.txt",
@@ -46,7 +46,7 @@ Use this local repo-only checklist before publishing mdkg:
46
46
 
47
47
  1. Confirm package intent and version in `package.json`, `package-lock.json`, `README.md`, `CLI_COMMAND_MATRIX.md`, and `CHANGELOG.md`.
48
48
  2. Use a clean npm cache: `export NPM_CONFIG_CACHE=/private/tmp/mdkg-npm-cache`.
49
- 3. Run `npm ci`, `npm run build`, `node scripts/assert-publish-ready.js`, `npm run test`, `npm run cli:check`, `node dist/cli.js validate`, `npm run smoke:consumer`, `npm run smoke:matrix`, `npm run smoke:upgrade`, `npm run smoke:init`, `npm run smoke:capabilities`, `npm run smoke:archive-work`, `npm run smoke:bundle`, `npm run smoke:bundle-import`, and `npm run smoke:visibility`.
49
+ 3. Run `npm ci`, `npm run build`, `node scripts/assert-publish-ready.js`, `npm run test`, `npm run cli:check`, `node dist/cli.js validate`, `npm run smoke:consumer`, `npm run smoke:matrix`, `npm run smoke:upgrade`, `npm run smoke:init`, `npm run smoke:capabilities`, `npm run smoke:archive-work`, `npm run smoke:bundle`, `npm run smoke:subgraph`, and `npm run smoke:visibility`.
50
50
  4. Run `npm pack --dry-run --json` and confirm the tarball includes `dist/cli.js`, compiled folders, `dist/init/`, release docs, and `scripts/postinstall.js`.
51
51
  5. Confirm registry state with `npm view mdkg version --registry=https://registry.npmjs.org/`.
52
52
  6. Publish only after the registry still shows the previous version and npm auth is known to have write access.
@@ -83,6 +83,7 @@ const VALUE_FLAGS = new Set([
83
83
  "--source-path",
84
84
  "--source-repo",
85
85
  "--max-stale-seconds",
86
+ "--requires",
86
87
  ]);
87
88
  const BOOLEAN_FLAGS = new Set([
88
89
  "--tolerant",
@@ -116,6 +117,7 @@ const BOOLEAN_FLAGS = new Set([
116
117
  "--with-scripts",
117
118
  "--clear-blocked-by",
118
119
  "--all",
120
+ "--fresh-only",
119
121
  ]);
120
122
  const FLAG_ALIASES = {
121
123
  "--o": "--out",
@@ -0,0 +1,44 @@
1
+ "use strict";
2
+ var __importDefault = (this && this.__importDefault) || function (mod) {
3
+ return (mod && mod.__esModule) ? mod : { "default": mod };
4
+ };
5
+ Object.defineProperty(exports, "__esModule", { value: true });
6
+ exports.atomicWriteFile = atomicWriteFile;
7
+ exports.writeFileExclusive = writeFileExclusive;
8
+ const crypto_1 = __importDefault(require("crypto"));
9
+ const fs_1 = __importDefault(require("fs"));
10
+ const path_1 = __importDefault(require("path"));
11
+ function randomSuffix() {
12
+ return `${process.pid}-${Date.now()}-${crypto_1.default.randomBytes(6).toString("hex")}`;
13
+ }
14
+ function writeAndSync(filePath, data, flags) {
15
+ const handle = fs_1.default.openSync(filePath, flags);
16
+ try {
17
+ if (typeof data === "string") {
18
+ fs_1.default.writeFileSync(handle, data, "utf8");
19
+ }
20
+ else {
21
+ fs_1.default.writeFileSync(handle, data);
22
+ }
23
+ fs_1.default.fsyncSync(handle);
24
+ }
25
+ finally {
26
+ fs_1.default.closeSync(handle);
27
+ }
28
+ }
29
+ function atomicWriteFile(filePath, data) {
30
+ fs_1.default.mkdirSync(path_1.default.dirname(filePath), { recursive: true });
31
+ const tempPath = path_1.default.join(path_1.default.dirname(filePath), `.${path_1.default.basename(filePath)}.${randomSuffix()}.tmp`);
32
+ try {
33
+ writeAndSync(tempPath, data, "wx");
34
+ fs_1.default.renameSync(tempPath, filePath);
35
+ }
36
+ catch (err) {
37
+ fs_1.default.rmSync(tempPath, { force: true });
38
+ throw err;
39
+ }
40
+ }
41
+ function writeFileExclusive(filePath, data) {
42
+ fs_1.default.mkdirSync(path_1.default.dirname(filePath), { recursive: true });
43
+ writeAndSync(filePath, data, "wx");
44
+ }
@@ -0,0 +1,72 @@
1
+ "use strict";
2
+ var __importDefault = (this && this.__importDefault) || function (mod) {
3
+ return (mod && mod.__esModule) ? mod : { "default": mod };
4
+ };
5
+ Object.defineProperty(exports, "__esModule", { value: true });
6
+ exports.withMutationLock = withMutationLock;
7
+ exports.lockTimeoutFromConfig = lockTimeoutFromConfig;
8
+ const fs_1 = __importDefault(require("fs"));
9
+ const path_1 = __importDefault(require("path"));
10
+ const HELD_LOCKS = new Set();
11
+ function sleepSync(ms) {
12
+ const shared = new SharedArrayBuffer(4);
13
+ const view = new Int32Array(shared);
14
+ Atomics.wait(view, 0, 0, ms);
15
+ }
16
+ function lockDir(root) {
17
+ return path_1.default.resolve(root, ".mdkg", "index", "write.lock");
18
+ }
19
+ function lockOwner() {
20
+ return JSON.stringify({
21
+ pid: process.pid,
22
+ node: process.version,
23
+ created_at: new Date().toISOString(),
24
+ }, null, 2);
25
+ }
26
+ function withMutationLock(root, timeoutMs, fn) {
27
+ const dir = lockDir(root);
28
+ if (HELD_LOCKS.has(dir)) {
29
+ return fn();
30
+ }
31
+ fs_1.default.mkdirSync(path_1.default.dirname(dir), { recursive: true });
32
+ const started = Date.now();
33
+ let lastError;
34
+ while (Date.now() - started <= timeoutMs) {
35
+ try {
36
+ fs_1.default.mkdirSync(dir);
37
+ let acquired = false;
38
+ try {
39
+ fs_1.default.writeFileSync(path_1.default.join(dir, "owner.json"), lockOwner(), "utf8");
40
+ HELD_LOCKS.add(dir);
41
+ acquired = true;
42
+ try {
43
+ return fn();
44
+ }
45
+ finally {
46
+ HELD_LOCKS.delete(dir);
47
+ fs_1.default.rmSync(dir, { recursive: true, force: true });
48
+ }
49
+ }
50
+ catch (err) {
51
+ if (!acquired) {
52
+ fs_1.default.rmSync(dir, { recursive: true, force: true });
53
+ }
54
+ throw err;
55
+ }
56
+ }
57
+ catch (err) {
58
+ const code = typeof err === "object" && err !== null && "code" in err ? String(err.code) : "";
59
+ if (code !== "EEXIST") {
60
+ throw err;
61
+ }
62
+ lastError = err;
63
+ sleepSync(25);
64
+ }
65
+ }
66
+ const detailPath = path_1.default.join(dir, "owner.json");
67
+ const owner = fs_1.default.existsSync(detailPath) ? fs_1.default.readFileSync(detailPath, "utf8").trim() : "unknown owner";
68
+ throw new Error(`timed out waiting for mdkg mutation lock at ${path_1.default.relative(root, dir)}; owner: ${owner || "unknown"}`);
69
+ }
70
+ function lockTimeoutFromConfig(config) {
71
+ return config.index.lock_timeout_ms;
72
+ }
package/package.json CHANGED
@@ -1,14 +1,14 @@
1
1
  {
2
2
  "name": "mdkg",
3
- "version": "0.1.2",
3
+ "version": "0.1.4",
4
4
  "description": "Markdown Knowledge Graph",
5
5
  "license": "MIT",
6
6
  "bin": {
7
7
  "mdkg": "dist/cli.js"
8
8
  },
9
9
  "scripts": {
10
- "build": "tsc -p tsconfig.build.json && node scripts/add-shebang.js && node scripts/copy-init-assets.js",
11
- "build:test": "tsc -p tsconfig.test.json",
10
+ "build": "node scripts/clean-build-output.js && tsc -p tsconfig.build.json && node scripts/add-shebang.js && node scripts/copy-init-assets.js",
11
+ "build:test": "node scripts/clean-build-output.js tests && tsc -p tsconfig.test.json",
12
12
  "test": "npm run build && npm run build:test && node --test dist/tests/**/*.test.js",
13
13
  "test:coverage": "npm run build && npm run build:test && node --test --experimental-test-coverage dist/tests/**/*.test.js",
14
14
  "smoke:consumer": "npm run build && node scripts/smoke-consumer.js",
@@ -18,16 +18,19 @@
18
18
  "smoke:capabilities": "npm run build && node scripts/smoke-capabilities.js",
19
19
  "smoke:archive-work": "npm run build && node scripts/smoke-archive-work.js",
20
20
  "smoke:bundle": "npm run build && node scripts/smoke-bundle.js",
21
- "smoke:bundle-import": "npm run build && node scripts/smoke-bundle-import.js",
21
+ "smoke:bundle-import": "npm run smoke:subgraph",
22
22
  "smoke:visibility": "npm run build && node scripts/smoke-visibility.js",
23
+ "smoke:sqlite": "npm run build && node scripts/smoke-sqlite.js",
24
+ "smoke:parallel": "npm run build && node scripts/smoke-parallel.js",
23
25
  "cli:snapshot": "npm run build && node scripts/cli_help_snapshot.js",
24
26
  "cli:check": "npm run build && node scripts/cli_help_snapshot.js --check",
25
27
  "prepack": "npm run build && node scripts/assert-publish-ready.js",
26
- "prepublishOnly": "npm run test && npm run cli:check && node dist/cli.js validate && npm run smoke:consumer && npm run smoke:matrix && npm run smoke:upgrade && npm run smoke:init && npm run smoke:capabilities && npm run smoke:archive-work && npm run smoke:bundle && npm run smoke:bundle-import && npm run smoke:visibility && node scripts/assert-publish-ready.js",
27
- "postinstall": "node scripts/postinstall.js"
28
+ "prepublishOnly": "npm run test && npm run cli:check && node dist/cli.js validate && npm run smoke:consumer && npm run smoke:matrix && npm run smoke:upgrade && npm run smoke:init && npm run smoke:capabilities && npm run smoke:archive-work && npm run smoke:bundle && npm run smoke:subgraph && npm run smoke:visibility && npm run smoke:sqlite && npm run smoke:parallel && node scripts/assert-publish-ready.js",
29
+ "postinstall": "node scripts/postinstall.js",
30
+ "smoke:subgraph": "npm run build && node scripts/smoke-subgraph.js"
28
31
  },
29
32
  "devDependencies": {
30
- "@types/node": "^18.19.0",
33
+ "@types/node": "^24.0.0",
31
34
  "typescript": "^5.4.0"
32
35
  },
33
36
  "files": [
@@ -46,7 +49,7 @@
46
49
  "LICENSE"
47
50
  ],
48
51
  "engines": {
49
- "node": ">=18"
52
+ "node": ">=24.15.0"
50
53
  },
51
54
  "repository": {
52
55
  "type": "git",
@@ -56,5 +59,6 @@
56
59
  "markdown",
57
60
  "knowledge-graph",
58
61
  "cli"
59
- ]
62
+ ],
63
+ "dependencies": {}
60
64
  }