wiki-graph 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (69) hide show
  1. package/README.md +10 -271
  2. package/dist/cli.d.ts +1 -0
  3. package/dist/cli.js +2983 -4
  4. package/dist/cli.js.map +1 -0
  5. package/{data → dist/data}/editor/text_compressor.jinja +17 -13
  6. package/{data → dist/data}/help/commands/archive/next.jinja +7 -2
  7. package/{data → dist/data}/help/commands/gc.jinja +2 -1
  8. package/{data → dist/data}/help/commands/legacy/migrate.jinja +6 -1
  9. package/{data → dist/data}/help/commands/legacy.jinja +3 -2
  10. package/dist/data/help/commands/library-predicate.jinja +296 -0
  11. package/dist/data/help/commands/library.jinja +235 -0
  12. package/{data → dist/data}/help/commands/maintenance/chapter/add.jinja +3 -3
  13. package/{data → dist/data}/help/commands/maintenance/chapter/list.jinja +4 -4
  14. package/{data → dist/data}/help/commands/maintenance/chapter/move.jinja +4 -4
  15. package/{data → dist/data}/help/commands/maintenance/chapter/remove.jinja +1 -1
  16. package/{data → dist/data}/help/commands/maintenance/chapter/reset.jinja +1 -1
  17. package/{data → dist/data}/help/commands/maintenance/chapter/set-source.jinja +2 -2
  18. package/{data → dist/data}/help/commands/maintenance/chapter/set-summary.jinja +1 -1
  19. package/{data → dist/data}/help/commands/maintenance/chapter/set-title.jinja +3 -3
  20. package/{data → dist/data}/help/commands/maintenance/chapter/tree.jinja +6 -6
  21. package/{data → dist/data}/help/commands/maintenance/chapter.jinja +10 -10
  22. package/{data → dist/data}/help/commands/maintenance/meta.jinja +7 -7
  23. package/dist/data/help/commands/maintenance/upgrade.jinja +35 -0
  24. package/dist/data/help/commands/maintenance.jinja +23 -0
  25. package/{data → dist/data}/help/commands/predicate.jinja +204 -28
  26. package/{data → dist/data}/help/commands/root.jinja +10 -3
  27. package/{data → dist/data}/help/commands/uri.jinja +165 -6
  28. package/{data → dist/data}/help/topics/config.jinja +10 -3
  29. package/dist/data/help/topics/library.jinja +87 -0
  30. package/{data → dist/data}/help/topics/readiness.jinja +32 -2
  31. package/{data → dist/data}/help/topics/recipe.jinja +42 -17
  32. package/{data → dist/data}/help/topics/runtime.jinja +20 -3
  33. package/{data → dist/data}/help/topics/uri.jinja +51 -25
  34. package/dist/gc-worker.d.ts +1 -0
  35. package/dist/gc-worker.js +548 -0
  36. package/dist/gc-worker.js.map +1 -0
  37. package/dist/index.cjs +12463 -19690
  38. package/dist/index.cjs.map +1 -1
  39. package/dist/index.d.cts +36 -842
  40. package/dist/index.d.ts +36 -842
  41. package/dist/index.js +13871 -9
  42. package/dist/index.js.map +1 -0
  43. package/dist/queue-worker.d.ts +1 -0
  44. package/dist/queue-worker.js +2110 -0
  45. package/dist/queue-worker.js.map +1 -0
  46. package/package.json +18 -45
  47. package/dist/cli.cjs +0 -41734
  48. package/dist/cli.cjs.map +0 -1
  49. package/docs/images/flowchart.excalidraw +0 -3293
  50. package/docs/images/flowchart.svg +0 -4
  51. package/docs/images/terminal-cn.png +0 -0
  52. package/docs/images/terminal-en.png +0 -0
  53. package/docs/wikispine-runtime.md +0 -143
  54. /package/{data → dist/data}/editor/clue_reviewer.jinja +0 -0
  55. /package/{data → dist/data}/editor/clue_reviewer_generator.jinja +0 -0
  56. /package/{data → dist/data}/editor/revision_feedback.jinja +0 -0
  57. /package/{data → dist/data}/guaranteed/response_intent_classifier.jinja +0 -0
  58. /package/{data → dist/data}/help/commands/maintenance/cover.jinja +0 -0
  59. /package/{data → dist/data}/help/commands/transform.jinja +0 -0
  60. /package/{data → dist/data}/help/topics/format.jinja +0 -0
  61. /package/{data → dist/data}/output/epub/cover.xhtml.jinja +0 -0
  62. /package/{data → dist/data}/output/epub/nav.xhtml.jinja +0 -0
  63. /package/{data → dist/data}/output/epub/package.opf.xml.jinja +0 -0
  64. /package/{data → dist/data}/output/epub/section.xhtml.jinja +0 -0
  65. /package/{data → dist/data}/topologization/book_coherence_extraction.jinja +0 -0
  66. /package/{data → dist/data}/topologization/chunk_extraction.jinja +0 -0
  67. /package/{data → dist/data}/topologization/evidence_choice.jinja +0 -0
  68. /package/{data → dist/data}/topologization/translate_chunks.jinja +0 -0
  69. /package/{data → dist/data}/topologization/user_focused_extraction.jinja +0 -0
@@ -25,7 +25,7 @@ Generation job behavior:
25
25
  - Reading Graph, Reading Summary, and Knowledge Graph generation run as device-local jobs.
26
26
  - Generation jobs consume LLM tokens, may incur provider charges, and can run for minutes to hours on large archives.
27
27
  - `wikg://local/job add` requires `--accept-cost` as an explicit cost/time acknowledgement.
28
- - Job records live under `~/.{{ commandName }}/jobs`.
28
+ - Job records live under `~/.wikigraph/jobs`.
29
29
  - `wikg://local/job add`, `wikg://local/job/<job-id> resume`, `wikg://local/job/<job-id> boost`, and `wikg://local/job/<job-id>/target set` try to start a transient worker.
30
30
  - The worker exits after idle time; it is not intended to be a permanent daemon.
31
31
  - `wikg://local/job/<job-id> watch --jsonl` is the stable agent-facing progress stream.
@@ -41,15 +41,30 @@ Progress and diagnostics:
41
41
  Workspace behavior:
42
42
  - Archive creation and EPUB import use temporary workspaces.
43
43
  - `transform --digest-dir <path>` uses the provided digest workspace for source digest runs.
44
- - Generation jobs use per-job directories under `~/.{{ commandName }}/jobs/work`, `~/.{{ commandName }}/jobs/cache`, and `~/.{{ commandName }}/jobs/logs`.
44
+ - Generation jobs use per-job directories under `~/.wikigraph/jobs/work`, `~/.wikigraph/jobs/cache`, and `~/.wikigraph/jobs/logs`.
45
45
  - Succeeded jobs delete their workspace by default.
46
46
  - Failed and canceled jobs keep their workspace for debugging until cleanup.
47
47
  - Job events, logs, and LLM cache are kept until the job record is cleaned.
48
48
 
49
+ Local state map:
50
+ - `~/.wikigraph/core.sqlite`: home registry and shared local state.
51
+ - `~/.wikigraph/cache/search-sessions.sqlite`: search query cache sessions used by paged search results.
52
+ - `~/.wikigraph/cache/continuation-cursors.sqlite`: opaque `wg next` continuation cursor records.
53
+ - `~/.wikigraph/cache/cache.sqlite`: Wikipedia page and other reusable local cache records.
54
+ - `~/.wikigraph/jobs`: durable generation job records, events, workspaces, caches, and logs.
55
+ - `~/.wikigraph/jobs/work`: per-job workspaces; failed and canceled jobs keep these for debugging.
56
+ - `~/.wikigraph/jobs/cache`: per-job LLM/cache artifacts kept with job records until cleanup.
57
+ - `~/.wikigraph/jobs/logs`: per-job diagnostic logs.
58
+ - `~/.wikigraph/staging`: archive coordinator state, materialized archive entries, local archive index staging, and library aggregate indexes.
59
+ - Search query caches and continuation cursors are local derived state; use `{{ commandName }} next --help` for cursor continuation and `{{ commandName }} gc --help` for cleanup.
60
+ - Do not manually delete active job, staging, or cursor state while commands are running; prefer `{{ commandName }} gc` and command-specific cleanup paths.
61
+
49
62
  In-place archive edits:
50
63
  - Mutating `.wikg` commands go through the archive coordinator.
51
64
  - Active generation jobs block destructive chapter maintenance edits for the same chapter.
52
- - Archive writes are coordinated through `~/.{{ commandName }}/staging`.
65
+ - Archive writes are coordinated through `~/.wikigraph/staging`.
66
+ - Library aggregate indexes are local derived state under staging and can be removed by cleanup when orphaned.
67
+ - Library cursors store the library index scope and re-check index freshness before continuing.
53
68
 
54
69
  Where to go next:
55
70
  - Need the normal workflow before debugging:
@@ -60,3 +75,5 @@ Where to go next:
60
75
  {{ commandName }} help format
61
76
  - Need URI command routing:
62
77
  {{ commandName }} help uri
78
+ - Need library registry, membership, path binding, or index behavior:
79
+ {{ commandName }} help library
@@ -8,7 +8,8 @@ Purpose:
8
8
  Use this page to choose a target, understand scope/object behavior, and select retrieval shape.
9
9
 
10
10
  Archive locator:
11
- - Archive/object command URIs must contain a `.wikg` locator.
11
+ - Normal archive/object command URIs contain a `.wikg` locator.
12
+ - Library URIs and local runtime URIs are separate URI families and do not contain `.wikg` locators.
12
13
  - Do not pass a bare filesystem path as a command target.
13
14
  /Users/me/book.wikg -> wikg:///Users/me/book.wikg
14
15
  ./book.wikg -> wikg://book.wikg
@@ -19,17 +20,29 @@ Archive locator:
19
20
  - Home archive locators may start with `wikg://~/`; `~` is expanded only in that leading form.
20
21
  - The first path segment ending in `.wikg` is the archive/object boundary.
21
22
 
23
+ Library locators:
24
+ - `wikg://lib` addresses the default library.
25
+ - `wikg://lib/<lib-id>` addresses a non-default library.
26
+ - `wikg://lib/arc/<archive-id>/` addresses one archive membership in the default library.
27
+ - `wikg://lib/<lib-id>/arc/<archive-id>/` addresses one archive membership in a non-default library.
28
+ - Add ordinary archive object suffixes after the trailing slash to query a managed archive:
29
+ wikg://lib/arc/<archive-id>/chapter
30
+ wikg://lib/arc/<archive-id>/entity
31
+ wikg://lib/arc/<archive-id>/triple
32
+ - Library-wide scopes such as `wikg://lib/entity` query the aggregate library index.
33
+ - Read `{{ commandName }} help library` before managing registry, membership, path binding, or library index lifecycle.
34
+
22
35
  Short and located URIs:
23
36
  Commands may return short object URIs without local file paths:
24
- wikg://chapter/3/title
25
- wikg://chapter/3/source#4..8
37
+ wikg://chapter/part/title
38
+ wikg://chapter/part/source#4..8
26
39
  wikg://chunk/12
27
40
  wikg://entity/Q8018
28
41
  wikg://triple/Q8018/discusses/Q123
29
42
  Returned short URIs are archive-relative handles, not complete command targets.
30
43
  Add the archive locator before passing them back to commands:
31
44
  wikg://book.wikg/entity/Q8018
32
- wikg:///books/book.wikg/chapter/3/source#4..8
45
+ wikg:///books/book.wikg/chapter/part/source#4..8
33
46
 
34
47
  Command routing:
35
48
  - `{{ commandName }} <scope-uri>` enumerates objects.
@@ -50,21 +63,27 @@ Common command shapes:
50
63
  {{ commandName }} <archive-uri> export --output-format <format>
51
64
  {{ commandName }} <archive-uri> inspect
52
65
  {{ commandName }} transform [options]
66
+ {{ commandName }} wikg://lib --help
67
+ {{ commandName }} wikg://lib/entity --query <query>
68
+ {{ commandName }} wikg://lib/arc/<archive-id>/entity --query <query>
53
69
  {{ commandName }} wikg://local/job
54
70
  {{ commandName }} wikg://local/job add --input <archive-uri|chapter-uri> --task <reading-graph|reading-summary|knowledge-graph> --accept-cost
55
71
  {{ commandName }} wikg://local/job/<job-id>
56
- {{ commandName }} <object-uri> related
57
- {{ commandName }} <object-uri> evidence
58
- {{ commandName }} <object-uri> pack --budget 5000
59
72
 
60
73
  Scope URIs:
61
74
  Scope URIs enumerate objects by default. With `--query <query>`, they apply keyword retrieval within that scope.
75
+ A chapter URI used as a scope covers that chapter's full subtree by default.
76
+ Add `--depth <N>` to limit chapter subtree expansion:
77
+ - no `--depth`: full subtree
78
+ - `--depth 0`: only the current chapter, or root-level chapters for `<archive-uri>/chapter`
79
+ - `--depth 1`: current/root-level chapters plus direct children
80
+ `--depth` must include a non-negative integer value; naked `--depth`, negative values, and non-integers are invalid.
62
81
  wikg://book.wikg
63
82
  wikg://book.wikg/chapter
64
- wikg://book.wikg/chapter/3
65
- wikg://book.wikg/chapter/3/chunk
66
- wikg://book.wikg/chapter/3/entity
67
- wikg://book.wikg/chapter/3/triple
83
+ wikg://book.wikg/chapter/part
84
+ wikg://book.wikg/chapter/part/chunk
85
+ wikg://book.wikg/chapter/part/entity
86
+ wikg://book.wikg/chapter/part/triple
68
87
  wikg://book.wikg/chunk
69
88
  wikg://book.wikg/entity
70
89
  wikg://book.wikg/triple
@@ -74,15 +93,15 @@ Scope URIs:
74
93
 
75
94
  Object URIs:
76
95
  Object URIs read one object by default.
77
- wikg://book.wikg/chapter/3/title
78
- wikg://book.wikg/chapter/3/state
79
- wikg://book.wikg/chapter/3/source
80
- wikg://book.wikg/chapter/3/source#4..8
81
- wikg://book.wikg/chapter/3/summary
82
- wikg://book.wikg/chapter/3/summary#4..8
83
- wikg://book.wikg/chapter/3/chunk/12
84
- wikg://book.wikg/chapter/3/entity/Q8018
85
- wikg://book.wikg/chapter/3/triple/Q8018/discusses/Q123
96
+ wikg://book.wikg/chapter/part/title
97
+ wikg://book.wikg/chapter/part/state
98
+ wikg://book.wikg/chapter/part/source
99
+ wikg://book.wikg/chapter/part/source#4..8
100
+ wikg://book.wikg/chapter/part/summary
101
+ wikg://book.wikg/chapter/part/summary#4..8
102
+ wikg://book.wikg/chapter/part/chunk/12
103
+ wikg://book.wikg/chapter/part/entity/Q8018
104
+ wikg://book.wikg/chapter/part/triple/Q8018/discusses/Q123
86
105
  wikg://book.wikg/chunk/12
87
106
  wikg://book.wikg/entity/Q8018
88
107
  wikg://book.wikg/entity/Q8018/wikipage
@@ -99,9 +118,12 @@ Triple pattern scopes:
99
118
  A complete triple URI with concrete subject, predicate, and object is an object URI.
100
119
 
101
120
  Object boundaries:
102
- - `chapter/<id>` is a scope, not an object.
103
- - `chapter/<id>/title` is the queryable chapter title object.
121
+ - `chapter/<path>` is a scope, not an object.
122
+ - `chapter/<path>/title` is the queryable chapter title object.
104
123
  - Whole `source` and `summary` objects are readable text streams, not collection scopes.
124
+ - Chapter-qualified scope URIs such as `<chapter-uri>/chunk`, `<chapter-uri>/entity`, and `<chapter-uri>/triple` expand to the chapter subtree and support `--depth`.
125
+ - Chapter-qualified resource and object URIs such as `/source`, `/summary`, `/title`, `/state`, `/chunk/<id>`, `/entity/<qid>`, and complete `/triple/<subject>/<predicate>/<object>` do not expand to subtrees and do not support `--depth`.
126
+ - `move`, `remove`, and `reset` are chapter node maintenance predicates; they target the chapter node itself, not the subtree. Use `remove --recursive` for explicit recursive deletion.
105
127
  - Ranged fragments such as `/source#4..8` and `/summary#4..8` are structured range objects.
106
128
  - Source and summary range numbers are 1-based and inclusive. `/source#1..2` means the first two source sentences.
107
129
  - Source text is the grounding material. `evidence` is not available for source, summary, or chapter scope URIs.
@@ -126,13 +148,13 @@ Lens selection:
126
148
  - `/chunk` targets Reading Graph chunks.
127
149
  - `/entity` targets grounded entities.
128
150
  - `/triple` targets Knowledge Graph relations.
129
- - `chapter/<id>` is a scope; `--query` over chapter titles returns `chapter/<id>/title` objects.
151
+ - `chapter/<path>` is a scope; `--query` over chapter titles returns `chapter/<path>/title` objects.
130
152
 
131
153
  Literal source passages:
132
154
  Use `--query` on the narrowest useful scope when the task is to find source wording or passages mentioning a literal term:
133
155
  {{ commandName }} <archive-uri> --query "term"
134
- {{ commandName }} <archive-uri>/chapter/<id> --query "term"
135
- {{ commandName }} <archive-uri>/chapter/<id>/source#20..30
156
+ {{ commandName }} <archive-uri>/chapter/<path> --query "term"
157
+ {{ commandName }} <archive-uri>/chapter/<path>/source#20..30
136
158
 
137
159
  Entity-grounded evidence:
138
160
  Use entity evidence when you already have an entity URI or QID and need grounded mentions, aliases, translations, variants, or source-backed support:
@@ -143,6 +165,7 @@ Entity-grounded evidence:
143
165
  Pagination:
144
166
  - Use default paging or `--limit <n>` while exploring.
145
167
  - Use `{{ commandName }} next <cursor>` to continue a paged result.
168
+ - Use `{{ commandName }} next <uri> <cursor>` when you want the cursor's stored index scope checked against the archive or library URI you expect.
146
169
  - Use `--all` only when you intentionally need every item from the selected scope.
147
170
  - Prefer the narrowest scope before using `--all`.
148
171
  - Read `{{ commandName }} help format` before choosing text, `--json`, `--jsonl`, or `--all --jsonl` for machine consumption.
@@ -158,6 +181,7 @@ Recovery hints:
158
181
  {{ commandName }} help readiness
159
182
  {{ commandName }} help runtime
160
183
  - URI rejected: convert bare filesystem paths to Wiki Graph URIs, and prepend the archive locator to short object URIs.
184
+ - Library URI rejected: non-default libraries use wikg://lib/<lib-id>; archive members use <lib-scope>/arc/<archive-id>.
161
185
  - Help did not appear: append `--help` to the URI or URI-bound predicate you are trying to inspect.
162
186
 
163
187
  Job URIs:
@@ -176,3 +200,5 @@ Related help:
176
200
  {{ commandName }} help format
177
201
  - Local jobs, cache, logs, and runtime debugging:
178
202
  {{ commandName }} help runtime
203
+ - Library registry, membership, path binding, and aggregate index:
204
+ {{ commandName }} help library
@@ -0,0 +1 @@
1
+ #!/usr/bin/env node