wiki-graph-core 0.3.1 → 0.4.1

package/README.md

@@ -0,0 +1,15 @@
+# Wiki Graph Core
+
+`wiki-graph-core` is the SDK package for [Wiki Graph](https://github.com/oomol-lab/wiki-graph), a long-text knowledge-base toolkit built around `.wikg` archives.
+
+Use it when you want to integrate Wiki Graph archive, document, retrieval, library, and runtime APIs directly from Node.js instead of using the `wg` CLI.
+
+```bash
+npm install wiki-graph-core
+# or
+pnpm add wiki-graph-core
+```
+
+Requires Node.js `>=22.12.0`.
+
+For the CLI package, install [`wiki-graph`](https://www.npmjs.com/package/wiki-graph). For full documentation, examples, source code, and issue tracking, see the [GitHub repository](https://github.com/oomol-lab/wiki-graph).

package/data/editor/text_compressor.jinja

@@ -26,11 +26,11 @@ Your compressed text must fall within the acceptable range. Going over the maxim
 - Natural expression (sound native in {{ user_language }}, not machine-translated)
 - Terminology consistency (technical terms and proper nouns should be accurately rendered)
 
-**Process:**
-1. Read and understand the marked content in whatever language(s) it uses
-2. Compress according to retention levels (`verbatim`, `detailed`, etc.)
-3. Translate the compressed result to {{ user_language }}
-4. Ensure the output reads naturally in {{ user_language }}
+Apply these constraints silently:
+- Understand the marked content in whatever language(s) it uses.
+- Compress according to retention levels (`verbatim`, `detailed`, etc.).
+- Translate the compressed result to {{ user_language }}.
+- Ensure the output reads naturally in {{ user_language }}.
 
 **Important Notes:**
 - The input text may be in any language or mix of languages (e.g., an English book with French quotes)
@@ -46,6 +46,14 @@ Your compressed text must fall within the acceptable range. Going over the maxim
 
 ---
 
+## Response Protocol (MUST FOLLOW)
+
+Return exactly one XML block matching this shape and nothing else: `<final>...</final>`
+
+The content inside `<final>` is the only user-visible compressed content. Do not put any extra text before or after this block. Do not use Markdown fences, headings, separators, explanations, processing notes, or prefaces.
+
+---
+
 ## Format Requirements (MUST FOLLOW)
 
 ### 1. Remove ALL `<chunk>` tags from your output
@@ -61,9 +69,9 @@ The input text contains `<chunk>` markup tags like this:
 - ❌ WRONG: `<chunk retention="detailed">比如刮风啊...</chunk>`
 - ✅ CORRECT: `比如刮风啊...`
 
-### 2. Output plain text only
+### 2. The content inside `<final>` must be plain text only
 
-Do NOT add:
+Inside `<final>`, do NOT add:
 - Section headers (like "## Compressed Text")
 - XML/HTML markup of any kind
 - Explanatory comments mixed with the content
@@ -89,7 +97,7 @@ Your output should read like a natural excerpt from a longer article.
 - Delete entire paragraphs if they don't have markup
 - Only keep ultra-short transitions if absolutely necessary (e.g., "三年后", "1344年", "此时")
 
-### Your compression strategy:
+### Compression priority:
 
 1. **First, preserve all marked content** (`<chunk>` tags) - this is your priority
 2. **Then, delete most unmarked text** - this is where you achieve compression
@@ -120,8 +128,4 @@ You are compressing a **segment** of a larger work (like part of a book chapter)
 
 ## Output Format
 
-[Optional: If you face difficult trade-offs, write 1-2 sentences about your approach. Keep this brief.]
-
----
-
-[Your compressed text - plain text only, no headers, no tags, written as continuous prose]
+Return exactly this XML shape with no Markdown code fence: `<final>Your compressed text - plain text only, no headers, no tags, written as continuous prose</final>`

package/data/help/commands/archive/next.jinja

@@ -10,20 +10,25 @@ Use when:
 
 Usage:
   {{ commandName }} next <cursor> [--limit <n>] [--json|--jsonl] [--help|-h]
-  {{ commandName }} next <archive-uri> <cursor> [--limit <n>] [--json|--jsonl] [--help|-h]
+  {{ commandName }} next <uri> <cursor> [--limit <n>] [--json|--jsonl] [--help|-h]
 
 Returns:
   - The next result page for the cursor.
 
 Notes:
   - Cursors are opaque short tokens stored in local Wiki Graph state.
-  - Passing `<archive-uri>` is optional; when provided, it must match the archive stored in the cursor.
+  - A cursor stores the index scope that produced it, such as a standalone archive index or a library index.
+  - Passing `<uri>` is optional; when provided, it is used only as a safety check and must match the cursor's stored index scope.
+  - Library cursors re-check that the library index is still current before continuing.
   - Use `--limit` to override the next page size.
 
 Examples:
   {{ commandName }} next c_7Kp9x2aQ
   {{ commandName }} next wikg:///books/book.wikg c_7Kp9x2aQ
+  {{ commandName }} next wikg://lib c_7Kp9x2aQ
 
 Related help:
   - URI scopes, pagination, JSON, and JSONL result shapes:
     {{ commandName }} help uri
+  - Library cursor scope and aggregate index freshness:
+    {{ commandName }} help library

package/data/help/commands/gc.jinja

@@ -21,10 +21,11 @@ Options:
 
 Managed areas:
   - Wiki Graph coordinator workspace files and cached SQLite databases.
-  - Search query cache.
+  - Search query cache and expired continuation cursor state under `~/.wikigraph/cache`.
   - Expired Wikipedia page cache.
   - Completed build job records and their durable event, cache, log, and workspace files.
   - Wiki Graph controlled temporary directories under the state directory.
+  - Orphaned library aggregate index staging that is no longer tied to a registered library.
 
 Output:
   Text output reports scanned resources, removed resources, and freed disk space.

package/data/help/commands/legacy/migrate.jinja

@@ -2,7 +2,10 @@ Help Type: command
 Command: {{ commandName }} legacy migrate
 
 Purpose:
-  Convert an `.sdpub` archive into the current `.wikg` format.
+  Deprecated alias for converting an `.sdpub` archive into the current `.wikg` format.
+
+Deprecated:
+  Use `{{ commandName }} maintenance upgrade <path.sdpub>` instead.
 
 Usage:
   {{ commandName }} legacy migrate <path.sdpub> [--output <path.wikg>]
@@ -16,5 +19,7 @@ Examples:
   {{ commandName }} legacy migrate ./book.sdpub --output ./book.wikg
 
 Related help:
+  - Current maintenance upgrade entry point:
+    {{ commandName }} maintenance upgrade --help
   - Current archive format, create/import, and export boundaries:
     {{ commandName }} help format

package/data/help/commands/legacy.jinja

@@ -2,18 +2,19 @@ Help Type: command
 Command: {{ commandName }} legacy
 
 Purpose:
-  Expose the single `.sdpub` upgrade command.
+  Deprecated compatibility entry point for `.sdpub` upgrade.
 
 Usage:
   {{ commandName }} legacy migrate <path.sdpub> [--output <path.wikg>]
 
 Commands:
-  migrate    Convert an `.sdpub` archive into the current `.wikg` format.
+  migrate    Deprecated alias for `{{ commandName }} maintenance upgrade <path.sdpub>`.
 
 Examples:
   {{ commandName }} legacy migrate ./book.sdpub
   {{ commandName }} legacy migrate ./book.sdpub --output ./book.wikg
 
 See Also:
+  {{ commandName }} maintenance upgrade --help
   {{ commandName }} legacy migrate --help
   {{ commandName }} help format

package/data/help/commands/library-predicate.jinja

@@ -0,0 +1,296 @@
+Help Type: command
+Command: {{ commandName }} {{ uri }} {{ predicate }}
+
+Library Predicate Command
+
+Target:
+  {{ uri }}
+
+Predicate:
+  {{ predicate }}
+
+{% if target.kind == "registry" and predicate == "add" %}
+Purpose:
+  Create a non-default library registry and create its empty library folder.
+
+Usage:
+  {{ commandName }} wikg://lib/registry add --path <folder> [--json]
+
+Notes:
+  - The folder must not already exist; the command creates it as an empty directory.
+  - Text output is the new public library URI. With `--json`, output includes the public id, URI, folder path, default marker, staging path, and timestamps.
+{% elif target.kind == "registry" and predicate == "list" %}
+Purpose:
+  List all library registries.
+
+Usage:
+  {{ commandName }} wikg://lib/registry [--json]
+{% elif target.kind == "path" and predicate == "get" %}
+Purpose:
+  Read this library folder path.
+
+Usage:
+  {{ commandName }} {{ uri }} [--json]
+{% elif target.kind == "path" and predicate == "set" %}
+Purpose:
+  Bind this library to an existing directory, then scan/reconcile `.wikg` files in that directory.
+
+Usage:
+  {{ commandName }} {{ uri }} set <folder> [--json|--jsonl]
+
+Notes:
+  - The command updates only this library registry's folder path.
+  - With `--jsonl`, it streams started/progress/completed events.
+{% elif target.kind == "archive-collection" and predicate == "add" %}
+Purpose:
+  Copy a `.wikg` archive into this library folder and register it as a managed archive member.
+
+Usage:
+  {{ commandName }} {{ uri }} add --input <path> [--to <relative-wikg-path>] [--json]
+{% elif target.kind == "archive-collection" and predicate == "scan" %}
+Purpose:
+  Reconcile this library registry with `.wikg` files currently under the library folder.
+
+Usage:
+  {{ commandName }} {{ uri }} scan [--json|--jsonl]
+
+Notes:
+  - With `--jsonl`, it streams started/progress/completed events.
+{% elif target.kind == "archive-tree" and predicate == "archive-tree" %}
+Purpose:
+  Show the archive member file tree.
+
+Usage:
+  {{ commandName }} {{ uri }} [--parent <relative-path>] [--depth <n>] [--json]
+
+Notes:
+  - `--parent` accepts a directory or concrete `.wikg` file and preserves ancestor nodes.
+  - `--depth` is calculated from the library scope root.
+  - This target is read-only and does not support `set` or `apply`.
+{% elif target.kind == "archive-path" and predicate == "get" %}
+Purpose:
+  Read this archive member's relative `.wikg` path.
+
+Usage:
+  {{ commandName }} {{ uri }} [--json]
+{% elif target.kind == "archive-path" and predicate == "set" %}
+Purpose:
+  Move or rename this managed archive inside the same library folder.
+
+Usage:
+  {{ commandName }} {{ uri }} set <relative-wikg-path> [--json]
+{% elif target.kind == "scope" and predicate == "create" %}
+Purpose:
+  Create a non-default library registry and create its empty library folder.
+
+Usage:
+  {{ commandName }} wikg://lib/registry add --path <folder> [--json]
+
+Notes:
+  - Create libraries from the registry collection, `wikg://lib/registry`.
+  - `<folder>` must not already exist; the command creates it as an empty directory.
+  - The folder is not scanned, and contained `.wikg` files are not registered in this stage.
+  - Registry data is stored in the global core database, not in the library folder.
+  - Text output is the new public library URI. With `--json`, output includes the public id, URI, folder path, default marker, staging path, and timestamps.
+  - Save the emitted URI or public id if you need to address this non-default library later.
+  - List all library registries with `{{ commandName }} wikg://lib/registry`.
+
+Examples:
+  {{ commandName }} wikg://lib/registry add --path ./research-library
+  {{ commandName }} wikg://lib/registry add --path ./research-library --json
+{% elif target.kind == "scope" and target.objectUri == "wikg://index" and predicate == "get" %}
+Purpose:
+  Read this library index status.
+
+Usage:
+  {{ commandName }} {{ uri }} [--json]
+
+Notes:
+  - The status tells whether the library index is current, dirty, missing, or disabled.
+  - Library-wide object `--query` and object scope enumeration require a current library index.
+  - Run `enable` when the status says the index needs to be built or rebuilt.
+{% elif target.kind == "scope" and target.objectUri == "wikg://index" and predicate == "enable" %}
+Purpose:
+  Build or rebuild this library's aggregate search index.
+
+Usage:
+  {{ commandName }} {{ uri }} enable [--jsonl]
+
+Notes:
+  - This is a full rebuild from registered present archives.
+  - Missing or unreadable library archive members are skipped and reported.
+  - Progress is emitted as text by default, or line-delimited events with `--jsonl`.
+  - `--json` is not supported for enable because progress is streamed.
+  - Library indexes are local derived state under Wiki Graph staging, not files inside the library folder.
+  - Rebuild holds the library write lock; concurrent `arc scan`, `arc add`, `path set`, `disable`, archive `path set`, or archive `remove` commands for the same library cannot run at the same time.
+  - Archive writes can mark the aggregate index dirty; rebuild again before relying on library-wide query when status reports dirty or stale.
+{% elif target.kind == "scope" and target.objectUri == "wikg://index" and predicate == "disable" %}
+Purpose:
+  Disable this library index and remove its current local materialization.
+
+Usage:
+  {{ commandName }} {{ uri }} disable [--json]
+
+Notes:
+  - This does not delete the library registry, metadata, folder, or managed `.wikg` archives.
+  - Library-wide object `--query` and object scope enumeration will fail until the index is enabled again.
+  - Library indexes do not support archive-style `embed` or `external` modes.
+  - Disable holds the library write lock and cannot run concurrently with membership changes or index rebuild for the same library.
+{% elif target.kind == "scope" and predicate == "add" %}
+Purpose:
+  Copy a `.wikg` archive into this library folder and register it as a managed archive member.
+
+Usage:
+  {{ commandName }} {{ uri }} add --input <path> [--to <relative-wikg-path>] [--json]
+
+Notes:
+  - `--input` is a normal filesystem path, not a `wikg://` archive URI.
+  - Without `--to`, the archive is copied to the library folder using its basename.
+  - `--to` must be a relative `.wikg` path inside the library folder.
+  - Parent directories for `--to` are created as needed.
+  - Existing target files are not overwritten.
+  - Adding an archive marks the library index dirty.
+  - The command holds the library write lock and cannot run concurrently with `arc scan`, library `path set`, archive path changes/removes, or index rebuild for the same library.
+{% elif target.kind == "scope" and predicate == "scan" %}
+Purpose:
+  Reconcile this library registry with `.wikg` files currently under the library folder.
+
+Usage:
+  {{ commandName }} {{ uri }} scan [--json]
+
+Notes:
+  - Scan recursively discovers `.wikg` files in the bound library folder.
+  - Registered files that are no longer present remain visible as missing/stale instead of being deleted silently.
+  - When a moved archive can be matched by its stable mutation token, scan preserves its archive id and updates the relative path.
+  - Scan marks the library index dirty when membership changes.
+  - Scan holds the library write lock and cannot run concurrently with membership changes or index rebuild for the same library.
+{% elif target.kind == "scope" and predicate == "get" %}
+Purpose:
+  Read this library scope.
+
+Usage:
+  {{ commandName }} {{ uri }} get [--json]
+
+Notes:
+  - For a bare library URI, this reads the same membership listing as the default command.
+  - For concrete library-wide objects such as `{{ uri }}/entity/<qid>`, use the object URI directly or ask that URI for help.
+  - This is not a list of all library registries; use known `wikg://lib/<lib-id>` URIs for non-default libraries.
+{% elif target.kind == "scope" and predicate == "list" %}
+Purpose:
+  This library scope does not expose an archive membership flat list.
+
+Usage:
+  {{ commandName }} {{ uri }} [--json]
+
+Notes:
+  - Use `{{ commandName }} {{ uri }}/arc/tree` to inspect the managed archive file tree.
+  - Use `{{ commandName }} {{ uri }}/arc scan` when the folder contents changed outside Wiki Graph.
+  - Use `{{ commandName }} wikg://lib/registry` to list all library registries.
+{% elif target.kind == "scope" and predicate == "remove" %}
+Purpose:
+  Remove this non-default library registry.
+
+Usage:
+  {{ commandName }} {{ uri }} remove --confirm [--json]
+
+Notes:
+  - The default library is system-managed and cannot be removed.
+  - Removing a library deletes its core registry record, metadata, membership records, and local derived index state.
+  - Removing a library does not delete the library folder or any `.wikg` files in it.
+  - `--confirm` is required.
+  - Remove holds the library write lock and cannot run concurrently with membership changes or index rebuild for the same library.
+{% elif target.kind == "archive" and predicate == "get" %}
+Purpose:
+  Read this library archive membership record.
+
+Usage:
+  {{ commandName }} {{ uri }} [--json]
+
+Notes:
+  - This reads membership information, not archive content.
+  - Add archive object suffixes such as `/chapter`, `/entity`, or `/triple` to query the managed archive contents.
+{% elif target.kind == "archive" and predicate == "remove" %}
+Purpose:
+  Delete this managed `.wikg` file from the library folder and unregister it.
+
+Usage:
+  {{ commandName }} {{ uri }} remove --confirm [--json]
+
+Notes:
+  - `--confirm` is required.
+  - If the registered file is already missing, the command can still clean the registry entry.
+  - This does not delete empty parent directories.
+  - Removing an archive marks the library index dirty.
+  - Remove holds the library write lock and cannot run concurrently with library scan, add, path binding, archive path changes, or index rebuild for the same library.
+{% elif target.kind == "metadata" and predicate == "set" %}
+Purpose:
+  Replace the whole library metadata map.
+
+Usage:
+  {{ commandName }} {{ uri }} set <json> [--json]
+  {{ commandName }} {{ uri }} set --json-input <json> [--json]
+  {{ commandName }} {{ uri }} set --input <path|-> [--json]
+
+Notes:
+  - `set` requires a JSON object.
+  - Metadata keys are free-form user fields.
+  - `--json` controls output shape; it is not JSON input.
+  - System registry fields such as id, public id, folder path, default marker, and staging path cannot be modified through metadata.
+  - With `--json`, success output is the resulting metadata map.
+{% elif target.kind == "metadata" and predicate == "get" %}
+Purpose:
+  Read this library metadata map.
+
+Usage:
+  {{ commandName }} {{ uri }} get [--json]
+
+Notes:
+  - This reads the whole metadata map.
+  - Metadata keys are free-form user fields.
+  - System registry fields such as id, public id, folder path, default marker, and staging path cannot be modified through metadata.
+{% elif target.kind == "metadata" and predicate == "put" %}
+Purpose:
+  Write one library metadata key.
+
+Usage:
+  {{ commandName }} {{ uri }} put <key> <value> [--json]
+  {{ commandName }} {{ uri }} put <key> --json-input <value> [--json]
+  {{ commandName }} {{ uri }} put <key> --input <path|-> [--json]
+
+Notes:
+  - Without JSON input, the value is stored as a string.
+  - With JSON input, the value is parsed as JSON.
+  - `--json` controls output shape; it is not JSON input.
+  - Metadata keys are command arguments, not URI segments.
+  - System registry fields such as id, public id, folder path, default marker, and staging path cannot be modified through metadata.
+{% elif target.kind == "metadata" and predicate == "delete" %}
+Purpose:
+  Delete one library metadata key.
+
+Usage:
+  {{ commandName }} {{ uri }} delete <key> [--json]
+
+Notes:
+  - Missing keys are ignored.
+  - With `--json`, success output is the resulting metadata map.
+{% elif target.kind == "metadata" and predicate == "clear" %}
+Purpose:
+  Clear this library metadata map.
+
+Usage:
+  {{ commandName }} {{ uri }} clear [--json]
+
+Notes:
+  - This clears user metadata only.
+  - It does not remove the library registry or change the library folder.
+{% else %}
+This library target does not support `{{ predicate }}`.
+{% endif %}
+
+Related help:
+  - {{ commandName }} {{ uri }} --help
+  - {{ commandName }} help library
+{% if target.kind == "scope" and target.objectUri == "wikg://index" %}
+  - {{ commandName }} help readiness
+{% endif %}
+  - {{ commandName }} help uri