mdkg 0.1.0 → 0.1.2

package/CHANGELOG.md

@@ -4,6 +4,99 @@ All notable changes to mdkg are documented here.
 
 This project follows a pragmatic changelog style inspired by Keep a Changelog. Versions use npm package versions.
 
+## 0.1.2 - 2026-05-19
+
+### Added
+
+- Added `.mdkg/index/capabilities.json` as a derived JSON cache for skills, `SPEC.md`, `WORK.md`, core docs, and design docs.
+- Added read-only `mdkg capability list/search/show` commands with JSON output, kind filters, and advisory visibility filters.
+- Added workspace `visibility` metadata for capability cache filtering, defaulting to `private`.
+- Added capability cache health reporting to `mdkg doctor`.
+- Added capability-cache smoke coverage for root plus child workspace aggregation and cache auto-rebuild.
+- Added packed-package init smoke coverage for fresh base init, fresh `mdkg init --agent`, removed flag failures, repeated init idempotency, doctor/validate, upgrade dry-run parity, task creation, and pack generation.
+- Added init preflight checks for seed config parseability and unmanaged skill mirror collisions.
+- Added first-class archive sidecars under `.mdkg/archive` with `mdkg archive add/list/show/verify/compress`.
+- Added deterministic single-file ZIP cache generation for archived source and artifact files.
+- Added `type: archive` graph nodes and `archive://<archive.id>` reference validation.
+- Added `mdkg work contract/order/receipt/artifact` lifecycle helpers for semantic mirror work contracts, work orders, receipts, and artifact registration.
+- Added archive/work packed-package smoke coverage for fresh temp repositories.
+- Added `mdkg bundle create/list/show/verify` for deterministic full `.mdkg` graph snapshot bundles.
+- Added private and public bundle profiles with fail-closed public filtering for private graph and archive refs.
+- Added bundle-local generated indexes (`global.json`, `skills.json`, `capabilities.json`) inside snapshot ZIPs.
+- Added bundle unit and CLI coverage plus packed-package bundle smoke coverage.
+- Added `mdkg bundle import add/list/rm/enable/disable/verify` for read-only child graph snapshot imports.
+- Added `bundle_imports` config with explicit alias, bundle path, visibility, expected profile, source metadata, and optional staleness policy.
+- Added `.mdkg/index/imports.json` as a derived import projection and health cache.
+- Added packed-package bundle import smoke coverage.
+- Added shared visibility policy enforcement for workspace nodes, archive sidecars, and imported bundle nodes.
+- Added `mdkg pack --visibility public|internal|private` for explicit public-safe and internal-safe packs.
+- Added `mdkg archive add --visibility private|internal|public` and `mdkg archive list --visibility ...`.
+- Added packed-package visibility smoke coverage.
+- Added `receipt_status: superseded` support for committed receipt mirrors.
+- Added `mdkg work receipt new|update --receipt-status superseded` CLI parity with graph validation.
+- Added runtime-style work/order/receipt fixture coverage with input refs, requested outputs, proof refs, artifacts, and hashes.
+- Added local qid support for `mdkg work order update`, `mdkg work receipt update`, and `mdkg work artifact add`.
+- Added `archive.large_cache_warning_bytes` config and `mdkg doctor` warnings for large committed archive ZIP caches.
+
+### Changed
+
+- `mdkg index` now writes the node index, skill index, and capability cache together.
+- Generated bootstrap config now includes the default capability cache path and root workspace visibility.
+- Docs and command matrix now teach capability discovery as separate from normal graph node search.
+- Made `mdkg init --agent` the single canonical AI-agent bootstrap path.
+- `mdkg init --agent` now creates `AGENTS.md` and `CLAUDE.md` alongside `AGENT_START.md`, `llms.txt`, `CLI_COMMAND_MATRIX.md`, default skills, event log, registry, and skill mirrors.
+- Init manifests are now mode-aware: base init only claims base assets, while agent init claims the full agent bootstrap.
+- Updated generated `.mdkg/README.md` onboarding to guide already-initialized repos toward `upgrade`, `new`, `search`, `pack`, and `validate`.
+- Added first-ingestion guidance to `AGENT_START.md` for imported docs bundles.
+- Hardened `WORK_ORDER.md` and `RECEIPT.md` templates with input refs, requested outputs, constraint refs, proof refs, attestation refs, and input/output hashes.
+- Init ignore updates now ignore raw archive source copies under `.mdkg/archive/**/source/` while leaving sidecar `.md` files and ZIP caches commit-eligible.
+- `mdkg doctor` now reports archive storage hygiene warnings for stray uncompressed archive files.
+- Added default bundle config under `.mdkg/bundles` without making `mdkg index` rewrite bundles.
+- Updated docs, command matrix, and release skills with bundle creation and verification guidance.
+- Updated seeded init/upgrade skills so managed workspaces receive pre-commit archive compression and private bundle refresh guidance.
+- `list`, `search`, `show`, `pack`, and `capability` now include enabled read-only bundle imports by default.
+- Imported nodes use import-alias qids such as `child_repo:task-1` and expose original bundle/source metadata in JSON output.
+- Stale imports warn during planning reads while `mdkg bundle import verify` exits nonzero.
+- Public bundle creation now fails when public local nodes reference private or internal imported graphs.
+- Public/internal bundle imports now require public bundle profiles.
+- `mdkg validate` and `mdkg doctor` now report public/internal references to less-visible mdkg records.
+- Archive JSON receipts now include sidecar visibility.
+- Archive sidecars created from outside-repo files now redact `source_path` to `external:<basename>` instead of storing absolute local paths.
+- `mdkg validate` and `mdkg archive verify` now share strict ZIP cache integrity checks for ZIP hash, readability, payload SHA-256, and payload byte size.
+- Work mirror docs and templates now state the canonical-system boundary for production order, receipt, feedback, dispute, payment, ledger, marketplace, fulfillment, and execution state.
+- Work lifecycle packed-package smoke now proves local qid mutation, order status updates, final superseded receipts, archive verification, indexing, show, and pack.
+
+### Fixed
+
+- Fixed fresh `mdkg init --agent` leaving missing managed wrapper docs that immediately required `mdkg upgrade --apply`.
+- Fixed misleading init summaries by reporting manifest, ignore, registry, event log, core pin, and skill mirror actions.
+- Fixed late init failure UX by printing a partial-init receipt with recovery guidance.
+- Kept bundle output deterministic across repeated creates when only `.mdkg/bundles/` changes.
+- Hardened publish readiness and init smoke checks to assert seeded release skills include archive compression and bundle refresh guidance.
+- Mutating task and work update flows now reject imported qids with explicit read-only import errors.
+- Work lifecycle mutation commands now reject imported order/receipt qids with explicit read-only bundle import guidance.
+- Local graph indexing now allows edges to configured import aliases without treating them as missing local workspace nodes.
+- Public bundle checks now reuse the same fail-closed policy as public/internal pack checks.
+- `mdkg archive verify --json` now emits a verification receipt for corrupt archive ZIP caches instead of being blocked by strict index validation.
+
+### Removed
+
+- Removed `mdkg init --llm`, `mdkg init --agents`, `mdkg init --claude`, and `mdkg init --omni`; each now fails before mutation with guidance to use `mdkg init --agent`.
+
+## 0.1.1 - 2026-05-12
+
+### Added
+
+- Added bundled template schema fallback so older workspaces can keep using graph inspection while missing newly introduced built-in templates.
+- Added `safe_to_apply`, `will_write_paths`, `preserved_customizations`, `blocking_conflicts`, and `apply_side_effects` fields to `mdkg upgrade --json`.
+
+### Changed
+
+- `mdkg new <built-in-type>` can use the installed package template when the workspace has not yet vendored that local template.
+- `mdkg doctor` and `mdkg validate` warn, rather than fail, when packaged fallback schemas cover missing local built-in templates.
+- `mdkg upgrade` human output now states whether the receipt is safe to apply and what paths would be written.
+- `mdkg upgrade` skips ignored event logs and points users to `mdkg event enable` instead of creating ignored `.mdkg/work/events/events.jsonl` files.
+
 ## 0.1.0 - 2026-05-12
 
 ### Added

package/README.md

@@ -11,9 +11,9 @@ mdkg stays deliberately boring:
 - repo-native under `.mdkg/`
 - TypeScript + Node.js 18+
 - zero runtime dependencies
-- no sqlite, daemon, hosted index, or vector DB
+- no required sqlite, daemon, hosted index, or vector DB
 
-Current package version in source: `0.1.0`
+Current package version in source: `0.1.2`
 
 ## The product shape
 
@@ -43,19 +43,13 @@ bun add -g mdkg
 
 Initialize mdkg in a repo:
 
-```bash
-mdkg init --llm
-```
-
-This is the generic OSS bootstrap path. It creates `.mdkg/` and updates `.gitignore` / `.npmignore` by default. Use `--no-update-ignores` to opt out of those ignore-file updates.
-
-Optional agent-ready scaffold:
-
 ```bash
 mdkg init --agent
 ```
 
-This adds strict-node `SOUL.md` / `HUMAN.md`, seeds the three default mdkg usage skills, creates `events.jsonl`, updates the skill registry, adds core pin updates, and creates mirrored skill folders under `.agents/skills/` and `.claude/skills/`.
+This is the canonical AI-agent bootstrap path. It creates `.mdkg/`, `AGENT_START.md`, `AGENTS.md`, `CLAUDE.md`, `llms.txt`, `CLI_COMMAND_MATRIX.md`, strict-node `SOUL.md` / `HUMAN.md`, the three default mdkg usage skills, `events.jsonl`, the skill registry, core pin updates, and mirrored skill folders under `.agents/skills/` and `.claude/skills/`. It also updates `.gitignore` / `.npmignore` by default. Use `--no-update-ignores` to opt out of those ignore-file updates.
+
+For a non-agent markdown graph only, run `mdkg init`.
 
 Preview safe scaffold upgrades in an existing mdkg workspace:
 
@@ -70,7 +64,9 @@ Apply only after reviewing the receipt:
 mdkg upgrade --apply
 ```
 
-Upgrade is intentionally conservative. It creates missing managed startup docs and updates unchanged mdkg seed assets, but preserves customized docs, templates, skills, and core files as reported conflicts. Agent-enabled workspaces also get safe default skill upgrades and skill mirror refreshes; non-agent workspaces do not gain skills, events, or mirrors implicitly.
+Upgrade is intentionally conservative. It creates missing managed startup docs and templates, updates unchanged mdkg seed assets, and preserves customized docs, templates, skills, and core files as reported preserved customizations. Review `safe_to_apply`, `will_write_paths`, and `apply_side_effects` in the JSON receipt before applying. Agent-enabled workspaces can receive safe default skill upgrades and skill mirror refreshes; ignored event logs are skipped with guidance to run `mdkg event enable` if provenance should be restored.
+
+Older workspaces can continue to inspect and validate current graph nodes before applying an upgrade. When local templates are missing for newly introduced built-in mdkg types, mdkg uses the installed package's bundled templates as a read-only schema fallback and warns that `mdkg upgrade --apply` can vendor the missing templates.
 
 Create a task:
 
@@ -98,14 +94,69 @@ Build deterministic context:
 ```bash
 mdkg pack task-1
 mdkg pack task-1 --profile concise --dry-run --stats
+mdkg pack task-1 --visibility public --dry-run
+```
+
+Create a full `.mdkg` graph snapshot bundle for root or child orchestration:
+
+```bash
+mdkg archive compress --all
+mdkg archive verify --json
+mdkg bundle create --profile private
+mdkg bundle verify .mdkg/bundles/private/all.mdkg.zip
+mdkg bundle list --json
 ```
 
+Bundles are explicit graph transport artifacts, separate from task context packs. Before a commit in repos that track archives or bundles, refresh compressed archive caches first, then create the private bundle so the committed graph state is self-consistent. Private bundles are the default and may be committed in private repos when configured. Public bundles require at least one selected workspace with `visibility: public` and include only public workspace content and public archive sidecars; bundle creation fails if public content points at private graph, archive, or imported bundle records.
+
+Import a child repo bundle as a read-only planning view:
+
+```bash
+mdkg bundle import add child_repo child-repo/.mdkg/bundles/private/all.mdkg.zip --source-path child-repo
+mdkg bundle import list --json
+mdkg search "child capability"
+mdkg show child_repo:work.example
+mdkg pack child_repo:work.example --dry-run --stats
+mdkg bundle import verify child_repo --json
+```
+
+Imported bundle nodes are projected under the import alias, for example `child_repo:task-1`. They are available to `list`, `search`, `show`, `pack`, and capability discovery, but remain read-only; mutate the child repo and refresh its bundle to change imported content. Stale imports warn during planning reads and fail `mdkg bundle import verify`. Public or internal imports must be backed by public bundle profiles; private imports stay private planning context.
+
 Validate before handoff or commit:
 
 ```bash
 mdkg validate
 ```
 
+Discover cached capability surfaces:
+
+```bash
+mdkg index
+mdkg capability list --kind skill --json
+mdkg capability search "image worker" --kind work --json
+mdkg capability show <id-or-qid-or-slug> --json
+```
+
+Register source and artifact files as committed archive sidecars:
+
+```bash
+mdkg archive add ./inputs/key_input_doc.pdf --id archive.key-input-doc --kind source --visibility private
+mdkg archive verify archive://archive.key-input-doc
+mdkg archive list --json
+```
+
+Create semantic mirror work contracts, orders, receipts, and artifacts:
+
+```bash
+mdkg work contract new "generate image" --id work.generate-image --agent-id agent.image-worker --kind image_generation --inputs prompt:text:required --outputs image_url:url:required
+mdkg work order new "generate image request" --id order.generate-image-1 --work-id work.generate-image --requester user://example --input-refs archive://archive.key-input-doc
+mdkg work receipt new "generate image receipt" --id receipt.generate-image-1 --work-order-id order.generate-image-1 --outcome success --receipt-status recorded
+mdkg work artifact add receipt.generate-image-1 ./outputs/image.png --id archive.generated-image --kind artifact
+```
+
+Receipt statuses are `recorded`, `verified`, `rejected`, and `superseded`.
+Update and artifact commands accept local ids or local qids; imported bundle qids are read-only and must be changed in their source workspace.
+
 Update structured task state and evidence while keeping body and narrative edits in markdown:
 
 ```bash
@@ -149,6 +200,9 @@ mdkg lives under a hidden root directory:
 - `.mdkg/work/` tasks, bugs, tests, epics, checkpoints
 - `.mdkg/templates/` templates used by `mdkg new`
 - `.mdkg/skills/` Agent Skills packages
+- `.mdkg/archive/` sidecar metadata plus deterministic compressed source/artifact caches
+- `.mdkg/bundles/` optional committed full graph snapshot bundles
+- `.mdkg/index/imports.json` generated read-only bundle import cache
 - `.agents/skills/` Codex/OpenAI-facing mirrored skills
 - `.claude/skills/` Claude-facing mirrored skills
 - `.mdkg/index/` generated cache files
@@ -164,6 +218,9 @@ These are the commands new users and agents should learn first:
 - `mdkg next`
 - `mdkg pack`
 - `mdkg skill`
+- `mdkg capability`
+- `mdkg archive`
+- `mdkg work`
 - `mdkg task`
 - `mdkg validate`
 
@@ -222,6 +279,19 @@ This repo now dogfoods three internal skills:
 
 Optional skill metadata with prefixes such as `ochatr_*` is treated as vendor extension data. Structured skill output exposes it under `extensions.ochatr` while keeping the top-level `ochatr` field as a compatibility alias introduced in 0.0.9. ochatr.ai is a pioneering adopter of this extension pattern, not the name of the base mdkg standard.
 
+## Capability cache
+
+mdkg maintains `.mdkg/index/capabilities.json` as a derived access cache for deterministic capability surfaces:
+- skills from `.mdkg/skills/**/SKILL.md`
+- `SPEC.md`
+- `WORK.md`
+- core docs
+- design docs
+
+The capability cache is not the full graph and is not source of truth. Normal tasks, epics, bugs, tests, feats, and checkpoints remain in the standard graph index. Markdown remains authoritative; deleting the cache is recoverable with `mdkg index` or by running a capability command when auto-reindex is enabled.
+
+Capability records aggregate enabled registered workspaces and include deterministic source metadata such as `workspace`, `visibility`, `kind`, `id`, `qid`, `path`, headings, refs, source hash, and `indexed_at`. Workspace `visibility` also feeds mdkg's export safety checks for public/internal packs and public bundles. This is a CLI safety layer, not secret scanning, body redaction, or a replacement for private git hosting.
+
 ## Agent workflow files
 
 mdkg recognizes a small set of canonical agent workflow documents:
@@ -235,13 +305,28 @@ Use `mdkg new spec|work|work_order|receipt|feedback|dispute|proposal "<title>"`
 
 Relational templates contain editable placeholder refs. `spec` and `work` scaffold as validation-clean standalone docs; `work_order`, `receipt`, `feedback`, `dispute`, and `proposal` need real refs before strict `mdkg validate` passes.
 
+For executable or purchasable capability mirrors, prefer the lifecycle helpers under `mdkg work ...`. They create and update `WORK.md`, `WORK_ORDER.md`, and `RECEIPT.md` semantic mirror files only. Production order state, receipt state, feedback, disputes, payments, ledgers, marketplace inventory, fulfillment records, and execution state remain canonical outside mdkg, such as in Postgres or another application database. Do not store raw secrets, credentials, live payment state, ledger mutations, canonical marketplace state, or bulky raw payloads in these mirrors.
+
+## Archive sidecars
+
+Archive entries live under `.mdkg/archive/<archive.id>/` and are normal graph nodes with `type: archive`. `mdkg archive add` copies the source into a managed local `source/` directory, writes a frontmatter sidecar `<file>.md`, and writes a deterministic single-file ZIP cache `<file>.zip`. The original source path is left untouched.
+
+Archive sidecars support `archive://archive.example` refs from orders, receipts, artifacts, proof refs, and other workflow metadata. `artifact://...` refs remain external or runtime-managed artifact identities; `archive://...` refs name committed mdkg archive sidecars. `mdkg validate` and `mdkg archive verify` both require the sidecar contract, ZIP cache hash, readable ZIP payload, payload SHA-256, and payload byte size to match. A missing raw local source copy is non-fatal when the committed sidecar and ZIP cache are valid.
+
+When the source passed to `mdkg archive add` is inside the repo, `source_path` is repo-relative. Outside-repo sources are redacted to `external:<basename>` so sidecars do not leak absolute local paths.
+
+Archive sidecar visibility defaults to `private`. Use `mdkg archive add --visibility public` only when the sidecar metadata and ZIP cache are safe for public packs or public bundles.
+
+By default, init/upgrade ignore generated raw archive source copies with `.mdkg/archive/**/source/`; sidecar `.md` files and compressed `.zip` caches remain commit-eligible. `mdkg doctor` warns when a committed archive ZIP cache exceeds `archive.large_cache_warning_bytes` in `.mdkg/config.json` (default `26214400`; set `0` to disable). Large-cache warnings do not block archive add or validation.
+
 ## Current direction
 
 This release includes:
 - `init --agent`
-- default ignore updates with `--no-update-ignores` for `.mdkg/index/` and `.mdkg/pack/`
+- default ignore updates with `--no-update-ignores` for `.mdkg/index/`, `.mdkg/pack/`, and raw archive source copies
 - root-only published init seed config
 - skills indexing and search/show/list support
+- JSON capability cache for skills, `SPEC.md`, `WORK.md`, core docs, and design docs
 - optional `skills: [...]` on work items
 - pack-time skill inclusion
 - latest-checkpoint resolver + index hint
@@ -250,16 +335,22 @@ This release includes:
 - agent workflow file types and semantic `mdkg new --id` support
 - product-specific skill mirrors for Codex/OpenAI and Claude
 - shared `AGENT_START.md` startup guidance
+- conservative `mdkg upgrade` with mode-aware init manifests
+- archive sidecars with deterministic ZIP caches
+- semantic mirror helpers under `mdkg work ...`
+- explicit public/internal/private visibility enforcement for packs, bundles, archives, imports, validation, and doctor diagnostics
+- strict archive ZIP payload integrity checks during validation
 
 Current direction:
-- keep the OSS story generic around `init --llm`
-- use `init --agent` for deeper AI-agent bootstrap
+- keep the OSS story generic around `mdkg init --agent`
+- use base `mdkg init` only for repos that do not want agent bootstrap assets
 - keep `pack <id>` at the center of the human/agent loop
 - use `mdkg task ...` for structured state changes and markdown edits for narrative/body content
 - make event logging guided instead of purely manual
 - dogfood real skills inside the repo
 - make skill authoring first-class through `mdkg skill`
 - make `CLI_COMMAND_MATRIX.md` the single source of truth for the live CLI surface
+- keep production execution databases canonical while mdkg stores committed semantic mirrors
 - run manual behavior audits before enforcing stronger coverage thresholds
 
 Design and decision records live in the internal graph under `.mdkg/design/`.
@@ -271,6 +362,8 @@ mdkg is not a secret store.
 Use these defaults:
 - keep `.mdkg/index/` gitignored
 - keep `.mdkg/pack/` gitignored
+- keep `.mdkg/archive/**/source/` gitignored unless a repo intentionally commits raw local copies
+- commit archive sidecar `.md` metadata and deterministic `.zip` caches when they are needed for reviewable evidence
 - event logs are committed by default; ignore or delete them manually if a repo wants local-only provenance
 - do not ship `.mdkg/` into production builds or published packages
 - if an external orchestrator is writing mdkg state, keep one durable writer per run and batch commits at end-of-run or checkpoint boundaries