@itradingai/aiwiki 0.2.16 → 0.2.19

package/docs/development-log.md

@@ -2,6 +2,233 @@
 
 This log records queue-driven AIWiki development milestones that should remain visible to future maintainers, not only in automation chat history.
 
+## 2026-06-08 - Base contract cleanup and safe optional directory pruning
+
+Status: implemented and locally verified, committed locally, blocked before GitHub push and npm publication. Test-server verification is now required before both GitHub push and npm publication.
+
+Version target: `@itradingai/aiwiki@0.2.19`
+
+Commit: `3ae25b9` (`Stabilize AIWiki base contract before public-trial work`)
+
+### Goal
+
+Make the base AIWiki experience quieter and more reliable before public-trial work. A new workspace should show the core knowledge workflow first, not a set of empty enhancement directories or legacy command paths. Agents should also have a safe, machine-readable way to detect and remove only empty optional directories.
+
+The scoped acceptance criteria were:
+
+- keep the main help and quick-start path focused on setup, Agent sync/check, ingest, context/query, lint, status, and doctor;
+- preserve legacy command behavior without presenting legacy commands as the primary path;
+- create only core directories by default in new workspaces;
+- create optional long-term directories only when ingest actually writes related outputs;
+- treat missing optional directories as normal in doctor and directory summaries;
+- report empty optional directories as safe-fix lint issues with stable JSON metadata;
+- make `lint --fix-empty-dirs --json` delete only known empty optional directories and empty known optional parent directories;
+- keep the base CLI out of crawling, vector search, RAG-over-wiki, RBAC, RSS, scheduled collection, and browser-plugin scope.
+
+### Implemented
+
+- `src/workspace.ts`
+  - Split core directories from optional enhancement directories.
+  - Made setup/init/doctor summaries core-first.
+  - Kept optional directories valid when they already exist, but stopped requiring them for a healthy workspace.
+
+- `src/payload.ts`, `src/ingest.ts`, and `src/wiki-entry.ts`
+  - Preserved `request.outputs` as an additive request instead of normalizing every ingest into all optional artifacts.
+  - Kept core ingest artifacts stable: Source Card, Wiki Entry, and Processing Summary.
+  - Created claims, assets, topics, and outline files only when payload content or explicit outputs require them.
+  - Omitted optional frontmatter links when the corresponding files are not written.
+
+- `src/lint.ts` and `src/app.ts`
+  - Added safe-fix metadata for empty known optional directories.
+  - Added `safe_fixes.available`, `safe_fixes.applied`, and `safe_fixes.only_safe_fixes` to JSON output.
+  - Added `aiwiki lint --fix-empty-dirs --json`.
+  - Kept deletion deliberately narrow: known optional empty directories only, never core directories, unknown directories, non-empty directories, or files.
+  - Reduced main help to the core user path while preserving existing command behavior.
+
+- Documentation and packaged skill files
+  - Updated README, usage docs, FAQ, Agent handoff, lint protocol, and skill instructions for the core-first workflow.
+  - Documented the Agent flow: run `aiwiki lint --json`, apply safe fixes only when allowed, rerun lint, and report the changed directories.
+  - Tightened package file inclusion so unrelated untracked docs are not accidentally packed.
+
+- Tests
+  - Updated workspace tests for the smaller default directory contract.
+  - Added ingest coverage for minimal output behavior.
+  - Added CLI coverage for `lint --fix-empty-dirs --json` and help/setup guidance.
+
+### Verification
+
+- `npm test`: passed, 59 tests.
+- `npm run release:check`: passed, including tests and `scripts/release-check.mjs`.
+- `npm pack --dry-run`: passed for `@itradingai/aiwiki@0.2.19`, 35 files, package size 77.1 kB, shasum `3f436503bb0dc3019b188941f06f3c518bbecc0b`.
+- Verification used `D:/Program Files/nodejs/npm.cmd` with a repo-local npm cache because the PowerShell `npm` shim points to a missing `C:/Users/Max/AppData/Roaming/npm/npm-cli.js`.
+
+### Release State
+
+The implementation is committed locally, but the GitHub push failed before publication:
+
+```text
+Warning: Identity file C:/Users/Max/.ssh/id_ed25519 not accessible: Permission denied.
+git@github.com: Permission denied (publickey).
+fatal: Could not read from remote repository.
+```
+
+Because GitHub push failed, npm publication was not attempted. The task queue and human board are marked `blocked`, and the Enterprise WeChat blocked notification was delivered successfully with HTTP 200 / `errcode:0`.
+
+### Testing Server Gate
+
+Test-server verification must run before updating GitHub or npm.
+
+The release gate for `0.2.19` is now a pre-GitHub and pre-npm tarball smoke test: create the exact local npm tarball that would be published, copy that `.tgz` to a task-specific directory on `170.106.73.197`, install it into a task-local Node project, and run the same smoke commands against that installed package. This proves the package artifact before it reaches GitHub or npm.
+
+The older published-package smoke test remains useful after npm publication as a final registry sanity check, but it is no longer allowed to be the first remote test. Pushing to GitHub or publishing to npm before test-server verification would make a public delivery surface the first real deployment surface, which is the wrong order for this queue.
+
+Required prepublication smoke commands:
+
+```bash
+aiwiki setup --path <tmp-vault> --yes
+aiwiki doctor --path <tmp-vault>
+inspect that only core directories are created by default
+aiwiki ingest-agent --payload <minimal-fixture> --path <tmp-vault>
+inspect that optional output directories remain absent unless needed
+aiwiki lint --json --path <tmp-vault>
+aiwiki lint --fix-empty-dirs --json --path <tmp-vault>
+aiwiki context <topic> --path <tmp-vault>
+aiwiki query <topic> --path <tmp-vault>
+```
+
+The smoke test must use only a task-specific temporary directory on the remote server. Do not install globally, do not reuse a real user vault, and do not clean outside the task directory.
+
+### Resume Steps
+
+Remote tarball smoke must pass first. Only after that, restore SSH key access for the current runtime user or configure GitHub authentication so this succeeds:
+
+```powershell
+git push origin main
+```
+
+Then continue the release chain:
+
+```powershell
+npm publish --access public
+npm view @itradingai/aiwiki version
+```
+
+After `npm view` returns `0.2.19`, a short published-package registry sanity check can be run, but the blocking test-server gate must already have passed before both GitHub push and npm publish.
+
+### Notes For Future Changes
+
+- Keep optional directories optional. Do not reintroduce required empty claims/assets/topics/outlines directories in new workspaces.
+- Keep safe fixes narrow and auditable. Do not let `--fix-empty-dirs` delete files, core directories, unknown directories, or non-empty directories.
+- Keep main help focused on the core path. Legacy commands can remain compatible without being promoted as first-run guidance.
+- Test-server verification from the local npm tarball must happen before GitHub push and before npm publication. Local pack verification alone is not enough to update GitHub or npm.
+
+## 2026-06-07 - Agent-first skill sync
+
+Status: implemented, locally verified, pushed to GitHub, blocked on npm OTP before publication.
+
+Version target: `@itradingai/aiwiki@0.2.18`
+
+Commit: `e41d634` (`Make Agent skill upgrades safe to sync automatically`)
+
+### Goal
+
+Make AIWiki skill upgrades safe for Agent-first operation. The intended user path is that a host AI Agent can upgrade AIWiki, synchronize its local skill instructions, tell the user what changed, and preserve a rollback path without requiring the user to manually inspect Agent home directories.
+
+The scoped acceptance criteria were:
+
+- provide one idempotent command for first install and future upgrades;
+- keep npm install side-effect free and avoid automatic writes to Agent home directories;
+- detect missing, current, changed, and unsupported Agent targets;
+- back up changed installed skill files before overwrite;
+- support `--dry-run` and `--json` for Agent-driven preview and automation;
+- improve CLI help, README, and Agent handoff docs for skill upgrade behavior;
+- update the packaged AIWiki skill so Agents know how to use the new context/query/lint surfaces.
+
+### Implemented
+
+- `src/app.ts`
+  - Added `aiwiki agent sync`.
+  - Added `aiwiki agent sync --agent <id> --yes`.
+  - Added `aiwiki agent sync --dry-run`.
+  - Added `aiwiki agent sync --json --yes`.
+  - Added `aiwiki agent check --json`.
+  - Added command-specific help for `aiwiki agent help`, `aiwiki agent sync --help`, `aiwiki context --help`, and `aiwiki query --help`.
+  - Added safe copy behavior that compares source and target files, no-ops when current, and writes timestamped backups before overwrite.
+  - Updated next-action guidance from manual install toward `aiwiki agent sync --yes`.
+
+- `skill/SKILL.md`
+  - Added skill version marker `aiwiki-skill-version: 0.2.18`.
+  - Added Agent-first setup and upgrade instructions.
+  - Documented `agent sync`, `agent check`, `--dry-run`, `--json`, and backup/rollback behavior.
+  - Expanded Agent guidance for filtered/explainable context JSON.
+
+- `skill/QUERY_PROTOCOL.md` and `skill/LINT_PROTOCOL.md`
+  - Documented filters, query scope, result quality, recommended next action, match reasons, quality signals, and lint/next handling for host Agents.
+
+- `skill/UPGRADE_NOTES.md`
+  - Added packaged notes that explain the current skill changes, expected Agent behavior after sync, and user-facing summary requirements.
+
+- `README.md`, `docs/USAGE.md`, and `docs/AGENT_HANDOFF.md`
+  - Added safe skill sync and upgrade instructions for first installs, long-gap upgrades, per-agent sync, JSON status checks, and rollback from backup.
+
+- `tests/cli.test.ts`
+  - Covered agent sync dry-run, install, JSON current state, changed-file backup, and overwrite behavior.
+  - Covered new help text and `agent check --json`.
+
+### Verification
+
+- `npm test`: passed, 56 tests.
+- `npm run release:check`: passed, including 56 tests and release-check.
+- `npm pack --dry-run`: passed for `@itradingai/aiwiki@0.2.18`.
+- Clean publish clone verification from `C:\Users\Max\AppData\Local\Temp\aiwiki-publish-dc800e0934564873853c9313d670122c`: `npm ci`, `npm run release:check`, and `npm pack --dry-run` passed.
+- Isolated smoke test with temporary `CODEX_HOME` and `CLAUDE_HOME`: passed for first install, `agent check --json`, changed skill backup, overwrite, and current-state dry-run.
+
+### Release State
+
+GitHub push succeeded:
+
+```text
+ceec9a1..e41d634 main -> main
+```
+
+npm publication from a clean clone is blocked by a one-time password challenge:
+
+```text
+npm error code EOTP
+npm error This operation requires a one-time password from your authenticator.
+```
+
+Current registry version remains `0.2.16`, so remote smoke tests for `0.2.18` have not run yet.
+
+### Resume Steps
+
+After npm OTP is available, publish from a clean committed tree or a fresh clean clone:
+
+```powershell
+npm publish --otp=<code>
+npm view @itradingai/aiwiki version
+```
+
+After `npm view` returns `0.2.18`, run remote smoke tests for:
+
+```bash
+aiwiki agent sync --dry-run
+aiwiki agent sync --json --yes
+aiwiki agent check --json
+aiwiki context --help
+aiwiki query --help
+```
+
+Use a task-specific temporary Agent home and vault on the remote server. Do not write to real user Agent homes during remote verification.
+
+### Notes For Future Changes
+
+- Keep `agent sync` explicit. Do not add npm lifecycle hooks that write into `CODEX_HOME`, `CLAUDE_HOME`, or other Agent home directories.
+- Preserve backup-before-overwrite behavior for any future Agent target.
+- Keep `agent install` compatible, but prefer `agent sync` in user-facing guidance because it handles both first install and upgrade.
+- Do not make the skill upgrade workflow depend on crawling, vector search, RAG-over-wiki, RBAC, RSS, scheduled collection, or browser plugins.
+- Package publication should use a clean clone or clean committed tree because unrelated untracked files can otherwise be included by broad package `files` rules.
+
 ## 2026-06-05 - AIWIKI-004 lint workbench
 
 Status: implemented, locally verified, pushed to GitHub, blocked on npm OTP before publication.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@itradingai/aiwiki",
-  "version": "0.2.16",
+  "version": "0.2.19",
   "type": "module",
   "description": "Agent-first AI knowledge base CLI for turning articles, links and notes into Obsidian-ready source cards, topics, outlines and reusable knowledge assets.",
   "license": "MIT",
@@ -38,7 +38,17 @@
     "LICENSE",
     "CONTRIBUTING.md",
     "SECURITY.md",
-    "docs/*.md",
+    "docs/20260607-aiwiki-feature-pruning-plan.md",
+    "docs/20260607-aiwiki-long-term-operating-roadmap.md",
+    "docs/AGENT_HANDOFF.md",
+    "docs/FAQ.md",
+    "docs/OBSIDIAN_DATAVIEW_PLAN.md",
+    "docs/README.md",
+    "docs/RELEASE.md",
+    "docs/ROADMAP.md",
+    "docs/SHOWCASE.md",
+    "docs/USAGE.md",
+    "docs/development-log.md",
     "docs/architecture.svg",
     "skill"
   ],

package/skill/LINT_PROTOCOL.md

@@ -9,27 +9,36 @@ Use this protocol when the user asks:
 
 ## Steps
 
-1. Call:
+1. Call JSON lint first:
 
 ```bash
-aiwiki lint
+aiwiki lint --json
 ```
 
-2. Read the terminal report.
-3. Mention the report path, usually:
+2. If `safe_fixes.only_safe_fixes` is true and the user allows cleanup, apply the built-in safe fix and rerun JSON lint:
+
+```bash
+aiwiki lint --fix-empty-dirs --json
+aiwiki lint --json
+```
+
+3. Read the terminal report.
+4. Mention the report path, usually:
 
 ```text
 dashboards/Lint Report.md
 ```
 
-4. Explain warnings and errors as structure-health feedback.
-5. Do not frame lint as "the user must manually audit every note".
+5. Explain warnings and errors as structure-health feedback.
+6. Do not frame lint as "the user must manually audit every note".
+7. If `aiwiki next` recommends `aiwiki agent sync --yes`, treat Agent skill setup as the next operational step before asking the user to ingest or query more material.
 
 ## Issue Meaning
 
 - `error`: broken structure such as a missing internal link.
 - `warning`: likely fix needed, such as missing source fields or stale fallback entries.
 - `info`: useful inventory, such as deterministic fallback count or duplicate titles.
+- `safe_fixes`: machine-readable count of safe fixes available/applied; `only_safe_fixes` means all current issues are safe to apply with the supported fixer.
 
 ## Repair Guidance
 
@@ -39,4 +48,4 @@ Prefer small, traceable fixes:
 - Fix broken wikilinks.
 - Add missing `source_card` or `raw_file` paths.
 - Ask the host Agent to provide `analysis` for scaffold entries.
-
+- Remove empty optional enhancement directories only through `aiwiki lint --fix-empty-dirs --json`; do not delete core directories, unknown directories, non-empty directories, or files.

package/skill/QUERY_PROTOCOL.md

@@ -16,11 +16,29 @@ Use this protocol when the user asks:
 aiwiki context "<topic>"
 ```
 
+Add filters when the user intent is specific:
+
+```bash
+aiwiki context "<topic>" --type wiki_entries --status active --limit 5
+aiwiki context "<topic>" --source-role input --wiki-type source_knowledge --limit 5
+aiwiki context "<topic>" --source-role output --limit 5
+aiwiki context "<topic>" --type source_cards --status to-review --limit 5
+```
+
 3. Read the JSON result.
 4. Prefer `matches.wiki_entries` before source cards, claims, topics, outlines, or raw refs.
-5. Pay attention to `generation_mode`, `quality`, and `warnings`.
-6. If the best Wiki Entry has `quality: "scaffold"`, tell the user that the entry is a traceable fallback and may need Agent enrichment.
-7. Do not scan `02-raw` by default unless:
+5. Read `query_scope` to understand what was searched.
+6. Read `result_quality` before deciding how confident the answer can be.
+7. Follow `recommended_next_action`:
+   - `use_matches_for_answer`: answer from the matches.
+   - `review_grounding_or_enrich_entry`: answer cautiously and suggest enrichment/review.
+   - `review_source_cards_then_create_wiki_entry`: explain that only source-level material exists.
+   - `broaden_query_or_ingest_source`: ask for a broader query or ingest more material.
+8. Use `match_reasons` to explain why an item was selected.
+9. Use `quality_signals` to decide whether the result is enriched, scaffold, needs grounding review, or only has weak source support.
+10. Use `related_refs` to mention useful local follow-up files.
+11. If the best Wiki Entry has `quality: "scaffold"` or `quality_signals` includes `grounding:needs_review`, tell the user that the entry is a traceable lead and may need Agent enrichment or evidence review.
+12. Do not scan `02-raw` by default unless:
    - Wiki Entries are insufficient.
    - The user asks to verify the original source.
    - There is a source conflict.
@@ -35,4 +53,3 @@ Include:
 - Source basis.
 - Known gaps or scaffold warnings.
 - Suggested next step.
-

package/skill/SKILL.md

@@ -3,12 +3,46 @@ name: aiwiki
 description: Agent-first AIWiki workflow for turning one URL/body into local Wiki knowledge files.
 ---
 
+<!-- aiwiki-skill-version: 0.2.19 -->
+
 # AIWiki Skill
 
 Use this skill when the user asks an Agent to process one URL, article body, or local text file with the `aiwiki` keyword, or says phrases like `入库 <url>` / `收录 <url>` / `从 AIWiki 里了解 <topic>`.
 
 AIWiki CLI does not fetch webpages and does not call an LLM. The host Agent reads and understands the source; AIWiki validates, writes, links, tracks, queries, and lints local Markdown knowledge files.
 
+## Agent-First Setup and Upgrade
+
+When the user asks you to install, update, or repair AIWiki Agent integration, prefer the idempotent sync command:
+
+```bash
+aiwiki agent sync --yes
+```
+
+For a specific host:
+
+```bash
+aiwiki agent sync --agent codex --yes
+aiwiki agent sync --agent claude --yes
+```
+
+Use `--dry-run` to preview without writing, and `--json` when you need machine-readable status:
+
+```bash
+aiwiki agent sync --agent codex --dry-run
+aiwiki agent sync --json --yes
+aiwiki agent check --json
+```
+
+Sync behavior:
+
+- missing installed skill: install the packaged AIWiki skill
+- current installed skill: leave unchanged
+- different installed skill: backup the old file, then overwrite with the packaged skill
+- unsupported host: do not write; use `aiwiki prompt agent` as a manual fallback
+
+After sync, tell the user the target path, any backup path, and that the target Agent may need to restart or reload before the new skill is active. Never edit Agent config during `npm install`; sync is the explicit safe step.
+
 ## Knowledge Base Purpose
 
 Before ingesting, querying, linting, or reorganizing material, read `_system/purpose.md` in the target AIWiki workspace when it exists. Treat it as the local contract for:
@@ -98,7 +132,7 @@ Because AIWiki CLI does not call an LLM, high-quality summary and knowledge extr
   },
   "request": {
     "mode": "ingest",
-    "outputs": ["source_card", "wiki_entry", "creative_assets", "topics", "draft_outline", "processing_summary"],
+    "outputs": ["source_card", "wiki_entry", "processing_summary"],
     "language": "zh-CN"
   }
 }
@@ -143,7 +177,26 @@ When the user asks to understand a topic from AIWiki, call:
 aiwiki context "<topic>"
 ```
 
-Use the returned JSON to answer. Prefer Wiki Entries first. Do not scan `02-raw` by default unless the Wiki result is insufficient, the user asks to verify the original text, or sources conflict.
+Use filters when the user's intent is clear:
+
+```bash
+aiwiki context "<topic>" --type wiki_entries --status active --limit 5
+aiwiki context "<topic>" --source-role output --limit 5
+aiwiki context "<topic>" --type source_cards --status to-review --limit 5
+```
+
+Use the returned JSON to answer. Prefer Wiki Entries first, but read these fields before responding:
+
+- `query_scope`: filters, limit, and searched groups
+- `result_quality`: total matches, best score, and whether a Wiki Entry was found
+- `recommended_next_action`: whether to answer, broaden the query, enrich, or review grounding
+- `match_reasons`: why each result matched
+- `quality_signals`: scaffold, enriched, grounding, status, and relationship hints
+- `related_refs`: local wikilinks and frontmatter relationships
+
+Do not scan `02-raw` by default unless the Wiki result is insufficient, the user asks to verify the original text, or sources conflict.
+
+If `quality_signals` contains `quality:scaffold` or `grounding:needs_review`, tell the user the result is a traceable lead that needs enrichment or review. Do not present it as final confirmed knowledge.
 
 For direct human terminal output, use:
 
@@ -155,13 +208,20 @@ aiwiki query "<topic>"
 
 ## Lint Protocol
 
-When the user asks to整理 / 检查 / lint the knowledge base, call:
+When the user asks to整理 / 检查 / lint the knowledge base, first call JSON lint:
+
+```bash
+aiwiki lint --json
+```
+
+If `safe_fixes.only_safe_fixes` is true and the user has allowed cleanup, apply the built-in safe fix and rerun JSON lint:
 
 ```bash
-aiwiki lint
+aiwiki lint --fix-empty-dirs --json
+aiwiki lint --json
 ```
 
-Explain warnings and errors as structure-health feedback. Do not frame lint as a requirement that the user manually review every item.
+Only `remove_empty_optional_dir` is auto-safe today. It removes known empty optional enhancement directories and must not delete core directories, unknown directories, non-empty directories, or files. Explain warnings and errors as structure-health feedback. Do not frame lint as a requirement that the user manually review every item.
 
 ## Source Role
 

package/skill/UPGRADE_NOTES.md

@@ -0,0 +1,22 @@
+# AIWiki Skill Upgrade Notes
+
+Use this note when an Agent finishes `aiwiki agent sync`.
+
+## Current Major Changes
+
+- Agent integration should use `aiwiki agent sync --yes` for both first install and upgrades.
+- Sync backs up changed installed skill files before overwriting them.
+- `aiwiki agent check --json` and `aiwiki agent sync --json --yes` provide machine-readable status for Agents.
+- `aiwiki context` supports `--type`, `--source-role`, `--wiki-type`, `--status`, and `--limit`.
+- Context JSON includes `query_scope`, `result_quality`, `recommended_next_action`, `match_reasons`, `quality_signals`, and `related_refs`.
+- Scaffold or grounding-review matches are traceable leads, not final confirmed knowledge.
+
+## Agent Reply After Sync
+
+Tell the user:
+
+- which Agent targets were installed, current, updated, or unsupported
+- where the new skill was written
+- where the old skill was backed up, if any
+- that the Agent may need a restart or reload
+- that rollback is possible by copying the backup file back to the target path